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Appendix A. 1: Logic Model for the Process and Impact Evaluation of MRC 


MINNESOTA READING CORPS (MRC) LOGIC MODEL (for Process & Impact Evaluation) 


Problem MRC & School Inputs/Resources 


Activities 


Short-term Outcomes 


Long-term Goals 


Students 
not school 
ready or 
reading 
below 
grade level 
in the State 
of 

Minnesota 


NOTE: A 
trained 
AmeriCorps 
Member 
(ACM) is 
both an input 
& beneficiary 
in the MRC 
model 


Selection of sites 
based on degree of 
student need & 
internal capacity to 
partner effectively 
with MRC 

School selection of 
“Tier 2* K-3 student 
participants based on 
assessment of 
current literacy skills 

School incorporation 
of web-based data 
management system 
to track & monitor 
student progress with 
literacy intervention 

School 

implementation of 
research-based core 
curriculum 


Recruitment, screening 
& placement of ACM 
by MRC and school 
staff 

Training of ACMs & 
Literacy Coaches in 
literacy interventions, 
assessment, 
data-driven 
decision-making & 
program rules (N=800) 

Identification & 
assignment of 
dedicated Literacy 
Coach at school to 
support & monitor 
ACMs 


NOTE: Contextual factors, such as 
community factors, school environment 
(principal, teachers, culture), family/individual 
risk & protective factors, influence the inputs, 
activities & outcomes 


Continuous improvement feedback loop among Master Coach, Internal Coach, ACMs to ensure implementation fidelity. 


Coaching 

Ongoing 
consultation with 
Master Coach 


I 

Supervision & 
coaching of ACMs 
by internal Coach 


• Monthly meeting 
with ACMs to 


review student 
progress & set 
individualized 
plans for each 
student 

• Ongoing 
observation & 
coaching to refine 
each ACMs' 
delivery of 
interventions 

• Fidelity 
assessment of 
ACM delivery of 
MRC 


Tutoring of Pre-K sudents by ACMs 

(N=20 per ACM per year) 

• Support classroom instruction (1:1; small 
group, class) 

• Enrich literacy environment 

• Chart progress of targeted students 

• Conduct monthly progress monitoring of 
targeted students 

• Assess all students 3x per year on literacy 
skills 

• Deliver 1:1 & small group literacty 
interventions to students needing extra 
assistance 


Tutoring of Pre-K sudents by ACMs 

(N=20 per ACM per year) 

• Assess students identified by site staff 
as needing assistance (i.e., closest to 
target target performance) 

• Deliver 1:1 tutoring to selected students 
(20 minutes.day, 5 days/week) 

• Chart weekly student progress in 
AIMSweb 

• Conduct weekly progress 
assessments & 3x per year 
benchmarking 

• Exit students deemed to be on track & 
add new students 


Other ACM initiated school-based 
volunteer activities (e.g. after school math 
tutoring, indepedent service projects) 



School practices 


• Implement core curricula 

• Implement MRC 
(supplemental intervention) 

• Schedule tutoring sessions 

• Designate Internal Coach 


Support implementation fidelity 
of MRC model 

Support coaching, assessment 
& benchmarking practices 
Integrate ACM into school 
setting 


System-level 

(Process Assessment) 

• Enriched classroom literacy 
environment (Pre-K only) 

• Expansion of early literacy 
goals in K-3 

• Increased integration of 
assessment practices into 
K-3 school culture 


Pre-K 

Achievement of 
“kindergarten 
ready" targets on 
IGDI & ELO 
measuers 


K-3 

Demonstrated 
proficiency on 
the state's 3rd 
grade reading 
test (MCA-II) 


ACMs 

Continued 
engagement 
with children's 
literacy 
education 

Continued 
involvement 
with national & 
community 
service 


Build internal 
capacity for 
RTI methods 
and literacy 
instruction 

Expand 
literacy goals 
site-wide 
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Appendix A.2: Illustration of the Randomization Process 
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Appendix A.3: Findings from Sensitivity Analysis 


The sensitivity analysis was performed by first classifying each student in the original sample as either included 
(coded as 1) or excluded (coded as 0) from the final analysis. Students were excluded from the analysis due to 
withdrawal from a sampled school or due to a significant amount of missing data resulting from chronic school 
absence. A logistic regression was then performed predicting the likelihood of inclusion in the final analysis. The 
analysis indicates that Black, Hispanic, and Other Race students were less likely to be included in the final analysis 
sample. Also, students assigned to the program group were less likely to be included in the final analysis than 
students in the control group. Although some demographic characteristics were more likely to be associated with the 
excluded students, these results should be viewed in the context of the baseline analysis (presented in Table 111.4. of 
the final report), which show that the baseline student demographics used in the final analysis were generally 
equivalent across program and control groups. 


Table A.3.1. Logistic regression predicting inclusion in final analysis sample 


Predictor 

Effect 

SE (Effect) 

Sig. 

Female 

-0.051 

(0.169) 


Dual Language Learner 

0.225 

(0.272) 


Black 

-1.410 

(0.227) 

*** 

Asian 

-0.187 

(0.361) 


Hispanic 

-1.405 

(0.347) 

*** 

Other race 

-2.383 

(0.400) 

*** 

Assigned program 

-0.992 

(0.183) 

*** 

Intercept 

3.932 

(0.309) 

*** 


*p <0.05 

**p <0.01 
***p< 0.001 


Sample includes 1,527 students 
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Appendix A.4: School Names Corresponding to School 
Codes 


Table A.4.1: School Names Corresponding to School Codes in Table 111,5 


Code 

School Name 

A 

Becker Primary 

B 

Kapoisa 

C 

Jackson Preparatory Magnet 

D 

F orest H ills 

E 

Northrup Urban Environmental Learning Center 

F 

Pine City Elementary 

G 

Sheridan Arts Magnet 

H 

Bryn Mawr Elementary 

1 Franklin Elementary 

J 

Frost Lake Magnet School of Technology & Global Studies 

K 

Green Central Park Elementary 

L 

Jefferson Elementary 

M 

Jenny Lind 

N 

Paul and Sheila Wellstone Elementary 

0 

Phalen Lake Hmong Studies Magnet 

P 

Riverside Central 

Q 

Bel Air Elementary 

R 

Nellie Stone J ohnson 

S 

Oakdale Elementary 

T 

Sunset T errace Elementary 
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Appendix A.5: Detailed Methodology 

As explained in section III.B of the report, three sets of analyses were conducted to answer the study's research 
questions: 

1. What is the impact of the M RC program on student literacy outcomes? 

a. Does the impact vary by student characteristics/demographics? 

b. Do assessment scores vary by AmeriCorps member characteristics/demographics? 

2. Does the impact of the program vary week to week? Does the number of weeks of intervention (i.e., dosage) 
impact student literacy outcomes? 

3. Does participation in MRC have a longer-term impact on student literacy outcomes as measured at the end of 
the school year? 

The first set of analyses focus on the Fall-Winter Experimental Study data collected during the first semester of the 
2012-2013 school year and was used to answer the first research question (RQ1), which assesses the impact of the 
MRC program on student literacy outcomes. The Fall-Winter experimental analyses employed growth models 
utilizing weekly progress monitoring data as primary outcomes and the Fall benchmarks as covariates to assess the 
impact ofthe MRC program on student literacy proficiency at each grade level. 

Because it was not possible to continue the experimental RCT throughout the entire school year, all students who 
were eligible atthe Winter benchmark to participate in the MRC program were allowed to receive services during the 
second semester of the 2012-2013 school year. Data from the first and second semester were combined to answer 
the second and third research questions aimed at assessing the week to week impact of the program on student 
literacy outcomes (RQ2), as well as the longer-term impact of the program on student literacy outcomes (RQ3). 

Analysis 1: Fall-Winter Experimental Analyses 
Impact Analysis 

The overall goal ofthe Fall-Winter Experimental Study was to obtain a measure of an average treatment effect of the 
MRC program by grade. In a simple experiment, this measure of impact would be a straightforward comparison of 
program (treatment) and control group means atthe end ofthe experimental period. However, because the 
evaluation team was able to obtain weekly assessment data on all students in the study, we were able to estimate 
the impact of the program on participants as a varying quantity over the weeks ofthe experimental period (Fall- 
Winter). This approach required that we estimate the effect of the MRC program on a weekly basis concurrent with 
typical growth trajectories. Therefore, our experimental analysis consisted of two phases for each grade. In the first 
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phase, a statistical model was developed to produce a set of regression coefficients that summarized the observed 
data. In the second phase, we then used the estimated model to predict marginal (average) scores for program 
participants and control group students at several weekly points. From these points, the evaluation team estimated 
the difference between program and control group students to measure program impacts. We then calculated the 
precision of our estimates for statistical tests. 

The modeling of the MRC data presented several challenges. First, as weekly assessments were measured during a 
non-linear developmental period, treatment effects had to be measured as offsets to typical growth patterns. This 
was accomplished by employing a difference-in-difference technique, whereby program participation moderated the 
estimated growth trajectory of students. Since growth tends to be non-linear during early childhood years, growth 
trajectories in kindergarten were also smoothed using quadratic time variables. 

Second, the distribution of the assessment metrics, especially in Kindergarten, tended to be highly skewed across 
the weeks of data, and so statistical models that assumed normality required log-transformed outcomes. Thus, we 
transformed all outcomes and benchmarks by taking the log of the score plus 1 (i.e., y*=ln(y+l)). Once the model 
was estimated, predicted scores, "xb,"were transformed back into the original metric with exp(xb)-l. 

Third, the analysis was further complicated by the nested nature of the data. Each observation is a weekly 
assessment nested within a student, who is in turn nested within a pair, which in turn is nested within a school. Thus, 
typical regression techniques would produce standard errors that may lead to false statements about effects. We 
instead employed mixed models or FH ierarchical Linear Modeling (FILM) that estimated several random effects to 
produce the appropriate standard errors. Finally, since these data were weekly measurements, there was likely to be 
temporal autocorrelation across each student's residuals. In other words, the score at week 2 was likely to be highly 
correlated with the score at week 1. Thus, our models employed autoregressive residuals to adjust the standard 
errors for this temporal correlation. 

Our final mixed models encompass all these features. In the model for each grade, the assessment from week w of 

student i in pairj in school k is a function of a set of effects for the trajectory, Y " , with autoregressive 

residuals. The intercept is then a function of the benchmark scores, b, an indicator of alternative treatments, a, and 
program participation, p. The intercept equation also includes random effects for the student, e, pair, u, and school, v. 
Finally, each of the q trajectory polynomials is modeled as a function of program participation, (3 a . + (3 a p. . Note 

that the outcome and benchmarks are transformed into a logged metric. 
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Thus, our final mixed model was 

/ 

ln [>W- + '] = n m + Xv" + P r *ij k 

9=1 

where 

Kotik = AjO/it + All,* ln |A* + 1 ] + Poz jk a ,jk + flmjkPtjk + + W yA- + V k 

and 

Kqijk ~ PqO T" Pq\P,jk 


We then estimated this model for each grade. Note that for grades 1, 2, and 3 that I =1, but for kindergarten I =2. To 
maintain generalizability, we employed school-level sampling weights in our estimation. The key parameters in these 

models are represented as Ai , which is the difference-in-difference effect. The estimated models are presented in 
Appendix B.l. 

The second phase of the impact analysis was to predict scores for program and control groups by week, and test the 
weekly differences statistically. Specifically, we estimated the outcomes (in the original score metric) for program and 
control students assuming average benchmarks and no alternative treatments 


v„. = 


exp 


Poo + Poi In t h + 1 ] + X ( P v0 ) 


9=1 

/ 


exp 


Poo + Pot ln + 1 ] + Pos + X ( PlO + Pq\ ) ^ 


-1 


9=1 


for control students 


for program participants 


What is revealing in the above expression is that the only differences between the control and program predictions 
are the terms J3 r - (which is the initial difference between program and control students) and P q \ (the difference-in- 
difference effects). Each point estimate had an estimated variance (i.e., sampling error) that was used to test whether 
the difference was 0. The variance of the estimates employed information about the model variance, so the statistical 
tests contrasting control and program estimates employed information about the precision of the model. 

Two types of subgroup analysis were employed by the evaluation team. First, performed an omnibus Chi-square test 
of the ' effects (the subgroup effects on the difference-in-difference coefficients) to determine whether subgroup 

membership impacted the program effects on student trajectory. If an impact was determined, the evaluation team 
employed the second phase of the subgroup analyses, in which we again produced predicted values and tested the 
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differences between program and control students. Forthese analyses, we included a set of subgroup indicators, S, 
that moderated the trajectory effects and the impact of program participation on the trajectory effects, 



ln [y mjk +'] = + X n qj y + 

9=1 

where 

ft Oijli ~ Poojk "*"A)1 jk ^ [Ayr jk a ijk PkHjkPjk ^ Pktsjk ^ijk ^ijk 11 jk V k 

s 

and 

^ qijk ft yO ft :/l Pyk + I/r, sjlc ^ ijk P ijk X Yqsjk^ijk 


Member- and School-Level Random Effects on Impacts 

In order to determine whether members or schools were influential on gains for those assigned treatment, a 
intraclass correlation (ICC or p) analysis was conducted. Intraclass correlations measure the portion of the total 
variation in an outcome that is associated with various levels of analysis. In this case, the member ICC, p member , 
measure the degree to which different students assigned the same member correlate on changes or gains in their 
assessment scores within a school. The school-level ICC, p SC hooh measures the degree to which students, who are 
located within the same school, but assigned to different members, correlate with one another. 

We estimate these parameters using a mixed model where the difference between the winter and fall benchmark 
score for the ith child assigned member j in school k is a function of an overall mean difference^, a member random 
effect, 4> ]k , a school random effect,^, and a within-member student random effect, e ijk . 

ylfk ~ y$k = a + fyk +tk + e ijk 


We then estimate the variances components Member var (<Pjk) ^ school var (Zk)’ and 

° student = var ( e ijk )> and calculate two ICCs 


The ICC for members is 


n ® member 

H member ~ 2 , 2 , 2 

° member school student 
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and for the school it is 


2 

n ° school 

" school — 2 , 2 , 2 

member school student 


A key component in evaluating ICCs is to measure the uncertainty associated with them. Recent work by Hedges, et 
al (2012) has provided a formula for the estimate of the variance of ICCs based on the estimated precision of the 
derived variance components. 1 

AmeriCorps Member Fixed Effects on Assessment Scores 

T o assess the impact of member fixed characteristics (i.e. observed demographics) on treatment students' 
assessment scores, the evaluation team estimated another statistical model on a set of data where the assessments 
were standardized within grade and combined into a single model. In other words, we estimated standardized 
assessment scores within each grade, combined the data, and fit a model predicting scores with member 
characteristics (age, education years, specialized degree, full time status, tenure status, race/ethnicity, and gender), 
a simple time trend, student gender, and the interaction between member and student gender. Since the outcomes 
are standardized, the effect is estimated from the change in standard deviations. Note that while we fit a multilevel 
model, the variance components for members are confounded with the variance attributed to grades and schools 
because members were assigned specific grades and schools. This required an additional set of models to test 
whether member random effects were substantial enough that they should be included in the final model. To answer 
whether members had unique random effects on student outcomes, we fit grade-specific unconditional models (i.e., 
no controls) and calculated the correlation of outcomes from students tutored by the same member. We found little 
variance attributed to random member effects and, since the resulting ICCs were small, member effects were not 
accounted for in the final model. 

Analysis 2: Full Year Non-Experimental Analysis 

For the second set of analyses, data from the firstand second semester were combined to answerthe second 
research question, which focused on better understanding patterns of week over week growth in reading proficiency 
by grade. After the first semester, all students who were eligible atthe Winter benchmark to participate in the MRC 
program were allowed to receive services (program and control group members). Therefore, while the evaluation 
team employed similar analytic coding strategies and estimation techniques as applied in the first analysis, the 
second analysis differed in that we acknowledged that variation in dosage existed over the course of the year. 


1 Hedges, Larry V., Hedberg, E . C ., and Kuyper, Arend M. 2012. The Variance of Intraclass Correlations in Three- and Four-Level Models. 
Educational and Psychological Measurement. 
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Therefore, our primary predictor was not program participation as a moderator of trajectory, but instead the number 
of cumulative weeks of receiving the program. 

The cumulative number of weeks of receiving the program was coded in such a way that for any given week, the 
value was the number of sessions each student received up to and including that week. For example, if a student 
never received sessions, their weekly value was always 0. If a student received sessions each week for the whole 
year, they were coded as 1 for the first week, 2 for the second week, etc. Also, when a student stopped receiving 
sessions, the final weeks ofthe year were coded the same as theirfinal week ofsessions. For example, if a student 
stopped receiving sessions at week 5, they were coded as 5 for the remaining weeks. This allowed individual student 
variation in the pattern ofsessions, while fitting a model that estimated the effect of receiving an additional week of 
sessions. 


Similar to our Fall-Winter experimental analysis, we again fit a multilevel model; however, we removed the random 
effect for pairs since other pair members were allowed the program after the first semester. In this model, we coded 
each week with the dichotomous variable for receiving the program, r, and used the cumulative sum of this variable 
as a predictor. As with our Fall-Winter experimental analysis, we again use polynomials to capture non-linearity in 
kindergarten. We also controlled for the semester, d, and original program assignment, p. The final statistical model is 


\n(y wlk "T 1) ik "F ft lik d-w + ^ ' ^qik "F ^ ' ^cik X twik I "F 


P r wik 


q=l 


c=l Vg=l 


where 


Phik - Poo k + PoikPik + e ik + v k 


where £^=i t wik is the number of cumulative sessions for that week. 


To summarize the results of this model, we estimated the tangential change in the outcome 

dy W ik 

9 Xg=l t wik 

when moving from 0 to 1 weeks ofsessions, 1 to 2 weeks ofsessions, and so on. We caution that these effects are 
not necessarily causal, since receiving treatment is a function of performance on benchmark. T o explain, if a student 
exceeds benchmarks in the Fall they qualify to exit the MRC program and stop receiving treatment. If, however, their 
performance is relatively close, but still below benchmark in the Winter they could re-enter the program. Thus, 
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negative associations between the number of sessions and student outcomes are likely due to assigning the program 
to struggling students rather than the program having causally detrimental effects. 

Analysis 3: Exploratory Analysis 

Analysis of Probabilities of Group Membership 

The ultimate goal of the program is to matriculate students into an "above benchmark" status. Therefore, our final 
analysis examines this question by asking which trajectory pattern are program participants most likely to follow. We 
conceived of two plausible methods to measure this. First, we consider nominal categories of trajectory patterns and 
estimate the likelihood of a student falling into one of these nominal categories based on the experimental 
assignment. Our second approach views trajectory as a more linear conceptthat is altered during and after 
exposure to the program. 

In the first approach, we coded each student as belonging to the following categories: 

■ No program effect (always below benchmark) 

■ T emporary program effect (moving above and below benchmark and ending the school year below) 

■ Final program effect (starting below benchmark, then eventually progressing and remaining above benchmark 
for the remainder of the school year) 

■ Always above benchmark 

■ Negative effect (above, then ending below benchmark) 

Of these five patterns, the latter two represented a very small portion of students (4) and were not included in the 
final analysis. We performed a multinomial logistic regression analysis predicting the likelihood of students falling into 
each of the three major classes or groups as a function of program assignment, cumulative sessions, and 
demographics. We also included an interaction term between program assignment and cumulative sessions, again 
because some students that where initially assigned to the control group received sessions in the second semester. 

Once we estimated the model (provided in Appendix D), the evaluation team then predicted the likelihood of following 
each group pattern for two average students who received the same dosage of the program, specifically 10 tutoring 
sessions, with the only difference being that one student was originally assigned to the program group and the other 
to the control group. The key comparison is the likelihood of falling into the 'final program effect" category for those 
students assigned to the program group versus the control group. Because all other differences between the two 
theoretical students were held constant in the model, including program dosage (i.e., 10 sessions of tutoring), the 
only difference is the timing in which the students would have received MRC tutoring (i.e., first vs. second semester). 
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Therefore, the findings from this analysis indicate whether early intervention of the program results in a higher or 
lower likelihood of falling into the "final program effect" group. 

Spline Analysis 

S pline models are typical in analyses that seek to fit a model to data measured over time, where specific events are 
hypothesized to change the trajectory. In our case, the spline models measure not only the growth of students over 
the course of the year, but specifically how those growth patterns are altered by participation in the program. The 
spline model allows for a linear trajectory across the entire school year, and also allows for this trajectory to be 
adjusted based on two phases: during treatment, D 1( and after treatment, D 2 . Thus, the spline model we fit to the 
data is 

y it = fa + /?i Week it + X Week lt ) + e it 

j 

The purpose of the analysis was to examine changes to the trajectory of student achievement prior to, during, and 
after receiving MRC tutoring, regardless of initial assignment to conditions (program or control group). We fit the 
model on a subset of cases in each grade that received at least one weekly session of tutoring during the school 
year. We also limited the analysis to only 10 weekly tutoring sessions to remove the cross-semester (i.e., first vs. 
second semester) effects of low performing students who would have required a larger dosage ofthe MRC 
intervention. 


APPENDIX A.5: DETAILED METHODOLOGY 


PAGE 12 


The Corporation for National and Community Service | 2014 


Appendix B.l: Fall-Winter Models and Effects Tables 


Table B .1.1 Model predicting Fall Kindergarten students' weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.635 

(0.020) 

*** 

Alternative treatment 

0.420 

(0.096) 

*** 

Program 

-0.306 

(0.096) 

*** 

Time b 

0.180 

(0.055) 

*** 

Program xtime 

0.180 

(0.009) 

*** 

Time squared 

-0.002 

(0.012) 

*** 

Program xtime squared 

-0.008 

(0.000) 

*** 

Intercept 

-0.096 

(0.001) 


Observations 




Weeks 

4474 



Students 

359 



Pairs 

193 



Schools 

17 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

** p < 0.01 
*** p <0.001 

Kindergarten sample includes 359 students. 
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Table B .1.2 Model predicting Fall first grade students' weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.482 

(0.011) 

*** 

Alternative treatment 

-0.053 

(0.020) 

** 

Program 

-0.024 

(0.014) 


Time b 

0.058 

(0.001) 

*** 

Program xtime 

0.008 

(0.001) 

*** 

Intercept 

1.649 

(0.038) 

*** 

Observations 




Weeks 

5078 



Students 

409 



Pairs 

215 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p< 0.001 

First grade sample includes 409 students. 
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Table B.1.3 Model predicting Fall second grade students' weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.693 

(0.017) 

*** 

Alternative treatment 

-0.057 

(0.026) 

* 

Program 

-0.044 

(0.018) 

* 

Time b 

0.033 

(0.001) 

*** 

Program xtime 

0.005 

(0.001) 

*** 

Intercept 

1.102 

(0.059) 

*** 

Observations 




Weeks 

3234 



Students 

265 



Pairs 

143 



Schools 

20 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p <0.001 

Second grade sample includes 265 students. 
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Table B .1.4 Model predicting Fall third grade students' weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.765 

(0.011) 

*** 

Alternative treatment 

-0.028 

(0.014) 


Program 

-0.047 

(0.009) 

*** 

Time b 

0.026 

(0.001) 

*** 

Program xtime 

0.004 

(0.001) 

*** 

Intercept 

0.983 

(0.045) 

*** 

Observations 




Weeks 

3214 



Students 

268 



Pairs 

156 



Schools 

20 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p<0.001 

Third grade sample includes 308 students. 
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Table B.1.5 Estimated scores for all Kindergarten program and control group students 


Kindergarten (N=359) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

1.931 

(0.150) 

*** 

1.636 

2.226 

Week 1 

Program 

1.484 

(0.126) 

*** 

1.237 

1.730 


Difference 

-0.447 

(0.127) 

*** 

-0.696 

-0.199 


Control 

3.855 

(0.222) 

*** 

3.420 

4.289 

Week 4 

Program 

5.281 

(0.284) 

*** 

4.724 

5.838 


Difference 

1.426 

(0.191) 

*** 

1.052 

1.800 


Control 

6.720 

(0.351) 

*** 

6.032 

7.408 

Week 7 

Program 

12.238 

(0.596) 

*** 

11.070 

13.407 


Difference 

5.518 

(0.406) 

*** 

4.723 

6.314 


Control 

10.787 

(0.538) 

*** 

9.734 

11.841 

Week 10 

Program 

22.255 

(1.050) 

*** 

20.198 

24.312 


Difference 

11.468 

(0.741) 

*** 

10.017 

12.919 


Control 

16.279 

(0.783) 

*** 

14.745 

17.812 

Week 13 

Program 

33.045 

(1.526) 

*** 

30.054 

36.037 


Difference 

16.767 

(1.065) 

*** 

14.679 

18.855 


Control 

23.318 

(1.175) 

*** 

21.016 

25.621 

Week 16 

Program 

40.540 

(1.978) 

*** 

36.663 

44.416 


Difference 

17.221 

(1.480) 

*** 

14.320 

20.122 
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Table B.1.6. Estimated scores for all first grade program participants and control students 


FirstGrade (N =409) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

24.100 

(0.272) 

*** 

23.566 

24.634 

Week 1 

Program 

23.581 

(0.259) 

*** 

23.074 

24.089 


Difference 

-0.518 

(0.328) 


-1.160 

0.124 


Control 

28.862 

(0.274) 

*** 

28.324 

29.399 

Week 4 

Program 

28.981 

(0.270) 

*** 

28.452 

29.509 


Difference 

0.119 

(0.315) 


-0.498 

0.736 


Control 

34.527 

(0.289) 

*** 

33.960 

35.094 

Week 7 

Program 

35.566 

(0.295) 

*** 

34.988 

36.143 


Difference 

1.039 

(0.316) 

** 

0.420 

1.658 


Control 

41.267 

(0.336) 

*** 

40.609 

41.926 

Week 10 

Program 

43.597 

(0.353) 

*** 

42.905 

44.290 


Difference 

2.330 

(0.368) 

*** 

1.608 

3.052 


Control 

49.286 

(0.436) 

*** 

48.432 

50.141 

Week 13 

Program 

53.393 

(0.470) 

*** 

52.472 

54.313 


Difference 

4.106 

(0.513) 

*** 

3.101 

5.112 


Control 

58.827 

(0.605) 

*** 

57.640 

60.013 

Week 16 

Program 

65.340 

(0.664) 

*** 

64.037 

66.642 


Difference 

6.513 

(0.771) 

*** 

5.002 

8.024 
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Table B.1.7 Estimated scores for all second grade program participants and control students 


Second Grade (N =265) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

29.951 

(0.457) 

*** 

29.055 

30.847 

Week 1 

Program 

28.280 

(0.435) 

*** 

27.428 

29.132 


Difference 

-1.671 

(0.487) 

*** 

-2.625 

-0.716 


Control 

33.170 

(0.464) 

*** 

32.260 

34.079 

Week 4 

Program 

31.856 

(0.451) 

*** 

30.971 

32.741 


Difference 

-1.314 

(0.469) 

** 

-2.233 

-0.395 


Control 

36.723 

(0.485) 

*** 

35.772 

37.674 

Week 7 

Program 

35.868 

(0.482) 

*** 

34.923 

36.814 


Difference 

-0.855 

(0.471) 


-1.778 

0.068 


Control 

40.646 

(0.529) 

*** 

39.608 

41.684 

Week 10 

Program 

40.371 

(0.536) 

*** 

39.320 

41.421 


Difference 

-0.275 

(0.512) 


-1.279 

0.728 


Control 

44.977 

(0.604) 

*** 

43.793 

46.162 

Week 13 

Program 

45.423 

(0.620) 

*** 

44.207 

46.639 


Difference 

0.446 

(0.610) 


-0.750 

1.641 


Control 

49.759 

(0.716) 

*** 

48.356 

51.162 

Week 16 

Program 

51.092 

(0.743) 

*** 

49.636 

52.549 


Difference 

1.334 

(0.772) 


-0.180 

2.847 
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Table B.1.8 Estimated scores for all third grade program participants and control students 


Third Grade (N=308) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

55.594 

(0.406) 

*** 

54.798 

56.390 

Week 1 

Program 

53.598 

(0.368) 

*** 

52.877 

54.319 


Difference 

-1.996 

(0.475) 

*** 

-2.927 

-1.066 


Control 

60.236 

(0.384) 

*** 

59.483 

60.989 

Week 4 

Program 

58.794 

(0.356) 

*** 

58.096 

59.492 


Difference 

-1.442 

(0.431) 

*** 

-2.287 

-0.598 


Control 

65.259 

(0.378) 

*** 

64.517 

66.001 

Week 7 

Program 

64.485 

(0.358) 

*** 

63.783 

65.186 


Difference 

-0.774 

(0.408) 


-1.573 

0.025 


Control 

70.693 

(0.401) 

*** 

69.908 

71.479 

Week 10 

Program 

70.717 

(0.384) 

*** 

69.964 

71.470 


Difference 

0.024 

(0.429) 


-0.816 

0.864 


Control 

76.574 

(0.460) 

*** 

75.672 

77.476 

Week 13 

Program 

77.543 

(0.444) 

*** 

76.672 

78.413 


Difference 

0.969 

(0.511) 


-0.033 

1.971 


Control 

82.936 

(0.560) 

*** 

81.839 

84.034 

Week 16 

Program 

85.018 

(0.541) 

*** 

83.956 

86.079 


Difference 

2.081 

(0.657) 

** 

0.793 

3.369 
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Table B.1.9. Model predicting gender-moderated Fall Kindergarten student weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.636 

(0.020) 

*** 

Alternative treatment 

0.424 

(0.096) 

*** 

Female 

-0.066 

(0.079) 


Program 

-0.421 

(0.081) 

*** 

Time b 

0.171 

(0.013) 

*** 

Program xtime 

0.207 

(0.018) 

*** 

Time squared 

-0.002 

(0.001) 

** 

Program xtime squared 

-0.009 

(0.001) 

*** 

Female x program 

0.221 

(0.110) 

* 

Female xtime 

0.017 

(0.017) 


Female x program xtime 

-0.051 

(0.024) 

* 

Female xtime squared 

0.000 

(0.001) 


Female x program xtime squared 

0.002 

(0.001) 


Intercept 

-0.061 

(0.076) 


Observations 




Weeks 

4474 



Students 

359 



Pairs 

193 



Schools 

17 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
*** p < 0.001 

Kindergarten sample includes 159 male and 200 female students 
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Table B.1.10. Model predicting gender-moderated Fall firstgrade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.482 

(0.011) 

*** 

Alternative treatment 

-0.053 

(0.020) 

** 

Female 

0.024 

(0.021) 


Program 

-0.038 

(0.021) 


Time b 

0.060 

(0.001) 

*** 

Program xtime 

0.010 

(0.002) 

*** 

Female x program 

0.031 

(0.029) 


Female xtime 

-0.003 

(0.002) 


Female x program xtime 

-0.003 

(0.003) 


Intercept 

1.636 

(0.039) 

*** 

Observations 




Weeks 

5078 



Students 

409 



Pairs 

215 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p< 0.001 

Firstgrade sample includes 195 male and 214 female students 
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Table B.1.11. Model predicting gender-moderated Fall third grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.692 

(0.017) 

*** 

Alternative treatment 

-0.049 

(0.026) 


Female 

0.026 

(0.025) 


Program 

-0.052 

(0.024) 

* 

Time b 

0.032 

(0.001) 

*** 

Program xtime 

0.010 

(0.002) 

*** 

Female x program 

0.032 

(0.036) 


Female xtime 

0.001 

(0.002) 


Female x program xtime 

-0.012 

(0.003) 

*** 

Intercept 

1.089 

(0.060) 

*** 

Observations 




Weeks 

3631 



Students 

308 



Pairs 

168 



Schools 

21 




Notes: a: outcome is natural log ofassessmentscore + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p< 0.001 

Third grade sample includes 165 male and 143 female students 
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Table B.1.12. Estimated scores for Kindergarten Male program participants and control students 


Kindergarten (N=159) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

2.012 

(0.191) 

*** 

1.636 

2.387 

Week 1 

Program 

1.333 

(0.141) 

*** 

1.057 

1.610 


Difference 

-0.678 

(0.187) 

*** 

-1.045 

-0.312 


Control 

3.861 

(0.257) 

*** 

3.357 

4.366 

Week 4 

Program 

5.158 

(0.315) 

*** 

4.540 

5.775 


Difference 

1.296 

(0.271) 

*** 

0.766 

1.827 


Control 

6.546 

(0.396) 

*** 

5.770 

7.322 

Week 7 

Program 

12.369 

(0.679) 

*** 

11.039 

13.699 


Difference 

5.823 

(0.554) 

*** 

4.737 

6.909 


Control 

10.263 

(0.594) 

*** 

9.100 

11.427 

Week 10 

Program 

22.875 

(1.216) 

*** 

20.491 

25.259 


Difference 

12.612 

(1.002) 

*** 

10.648 

14.576 


Control 

15.166 

(0.842) 

*** 

13.515 

16.816 

Week 13 

Program 

34.074 

(1.766) 

*** 

30.613 

37.535 


Difference 

18.908 

(1.444) 

*** 

16.079 

21.738 


Control 

21.310 

(1.294) 

*** 

18.774 

23.847 

Week 16 

Program 

41.384 

(2.341) 

*** 

36.797 

45.971 


Difference 

20.074 

(2.060) 

*** 

16.036 

24.111 
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Table B.1.13. Estimated scores for Kindergarten Female program participants and control students 


Kindergarten (N=200) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

1.867 

(0.169) 

*** 

1.535 

2.198 

Week 1 

Program 

1.637 

(0.156) 

*** 

1.330 

1.943 


Difference 

-0.230 

(0.173) 


-0.569 

0.109 


Control 

3.850 

(0.243) 

*** 

3.373 

4.326 

Week 4 

Program 

5.406 

(0.323) 

*** 

4.774 

6.038 


Difference 

1.556 

(0.260) 

*** 

1.046 

2.066 


Control 

6.865 

(0.392) 

*** 

6.096 

7.633 

Week 7 

Program 

12.129 

(0.656) 

*** 

10.845 

13.414 


Difference 

5.265 

(0.515) 

*** 

4.255 

6.274 


Control 

11.226 

(0.612) 

*** 

10.025 

12.426 

Week 10 

Program 

21.701 

(1.138) 

*** 

19.472 

23.931 


Difference 

10.476 

(0.902) 

*** 

8.708 

12.244 


Control 

17.218 

(0.904) 

*** 

15.447 

18.989 

Week 13 

Program 

32.114 

(1.644) 

*** 

28.892 

35.336 


Difference 

14.896 

(1.288) 

*** 

12.371 

17.421 


Control 

25.023 

(1.414) 

*** 

22.251 

27.795 

Week 16 

Program 

39.749 

(2.222) 

*** 

35.393 

44.105 


Difference 

14.726 

(1.917) 

*** 

10.968 

18.484 
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Table B.1.14. Estimated scores for first grade Male program participants and control students 


FirstGrade (N=195) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

23.830 

(0.376) 

*** 

23.094 

24.567 

Week 1 

Program 

23.022 

(0.334) 

*** 

22.368 

23.676 


Difference 

-0.809 

(0.469) 


-1.727 

0.110 


Control 

28.695 

(0.373) 

*** 

27.964 

29.426 

Week 4 

Program 

28.575 

(0.344) 

*** 

27.900 

29.250 


Difference 

-0.120 

(0.457) 


-1.016 

0.777 


Control 

34.512 

(0.388) 

*** 

33.752 

35.273 

Week 7 

Program 

35.412 

(0.373) 

*** 

34.681 

36.142 


Difference 

0.900 

(0.467) 


-0.016 

1.816 


Control 

41.469 

(0.453) 

*** 

40.581 

42.357 

Week 10 

Program 

43.829 

(0.450) 

*** 

42.946 

44.711 


Difference 

2.360 

(0.551) 

*** 

1.280 

3.440 


Control 

49.789 

(0.603) 

*** 

48.607 

50.970 

Week 13 

Program 

54.192 

(0.614) 

*** 

52.988 

55.395 


Difference 

4.403 

(0.767) 

*** 

2.900 

5.906 


Control 

59.738 

(0.861) 

*** 

58.050 

61.426 

Week 16 

Program 

66.950 

(0.895) 

*** 

65.197 

68.704 


Difference 

7.212 

(1.148) 

*** 

4.962 

9.463 
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Table B.1.15. Estimated scores for first grade Female program participants and control students 


FirstGrade (N=214) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

24.344 

(0.356) 

*** 

23.646 

25.042 

Week 1 

Program 

24.205 

(0.360) 

*** 

23.499 

24.911 


Difference 

-0.139 

(0.470) 


-1.060 

0.782 


Control 

29.020 

(0.350) 

*** 

28.335 

29.705 

Week 4 

Program 

29.431 

(0.363) 

*** 

28.719 

30.144 


Difference 

0.411 

(0.451) 


-0.473 

1.295 


Control 

34.559 

(0.358) 

*** 

33.857 

35.261 

Week 7 

Program 

35.741 

(0.385) 

*** 

34.988 

36.495 


Difference 

1.183 

(0.452) 

** 

0.298 

2.068 


Control 

41.120 

(0.411) 

*** 

40.314 

41.925 

Week 10 

Program 

43.360 

(0.455) 

*** 

42.468 

44.252 


Difference 

2.240 

(0.523) 

*** 

1.216 

3.264 


Control 

48.891 

(0.539) 

*** 

47.834 

49.947 

Week 13 

Program 

52.558 

(0.610) 

*** 

51.362 

53.754 


Difference 

3.667 

(0.718) 

*** 

2.259 

5.075 


Control 

58.096 

(0.762) 

*** 

56.603 

59.588 

Week 16 

Program 

63.663 

(0.875) 

*** 

61.949 

65.378 


Difference 

5.568 

(1.066) 

*** 

3.479 

7.656 
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Table B.1.16. Estimated scores for third grade Male program participants and control students 


Third Grade (N=165) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

55.957 

(0.549) 

*** 

54.880 

57.034 

Week 1 

Program 

53.882 

(0.448) 

*** 

53.005 

54.759 


Difference 

-2.075 

(0.658) 

** 

-3.364 

-0.786 


Control 

60.195 

(0.506) 

*** 

59.204 

61.186 

Week 4 

Program 

59.323 

(0.428) 

*** 

58.484 

60.162 


Difference 

-0.872 

(0.598) 


-2.044 

0.299 


Control 

64.749 

(0.485) 

*** 

63.798 

65.700 

Week 7 

Program 

65.303 

(0.427) 

*** 

64.467 

66.140 


Difference 

0.554 

(0.567) 


-0.557 

1.665 


Control 

69.642 

(0.508) 

*** 

68.647 

70.637 

Week 10 

Program 

71.877 

(0.459) 

*** 

70.978 

72.776 


Difference 

2.235 

(0.595) 

*** 

1.068 

3.402 


Control 

74.899 

(0.587) 

*** 

73.749 

76.049 

Week 13 

Program 

79.102 

(0.537) 

*** 

78.050 

80.154 


Difference 

4.203 

(0.706) 

*** 

2.819 

5.587 


Control 

80.547 

(0.724) 

*** 

79.128 

81.967 

Week 16 

Program 

87.044 

(0.667) 

*** 

85.737 

88.350 


Difference 

6.496 

(0.901) 

*** 

4.731 

8.262 
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Table B.1.17. Estimated scores for third grade Female program participants and control students 


Third Grade (N=143) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

55.284 

(0.534) 

*** 

54.238 

56.331 

Week 1 

Program 

53.168 

(0.544) 

*** 

52.102 

54.233 


Difference 

-2.117 

(0.718) 

** 

-3.523 

-0.710 


Control 

60.309 

(0.500) 

*** 

59.329 

61.289 

Week 4 

Program 

57.981 

(0.510) 

*** 

56.980 

58.981 


Difference 

-2.328 

(0.658) 

*** 

-3.618 

-1.039 


Control 

65.782 

(0.488) 

*** 

64.825 

66.739 

Week 7 

Program 

63.221 

(0.499) 

*** 

62.243 

64.199 


Difference 

-2.561 

(0.629) 

*** 

-3.794 

-1.328 


Control 

71.744 

(0.518) 

*** 

70.728 

72.760 

Week 10 

Program 

68.927 

(0.529) 

*** 

67.890 

69.965 


Difference 

-2.817 

(0.664) 

*** 

-4.118 

-1.516 


Control 

78.238 

(0.606) 

*** 

77.051 

79.426 

Week 13 

Program 

75.141 

(0.617) 

*** 

73.931 

76.350 


Difference 

-3.097 

(0.787) 

*** 

-4.640 

-1.554 


Control 

85.312 

(0.755) 

*** 

83.832 

86.792 

Week 16 

Program 

81.906 

(0.766) 

*** 

80.404 

83.408 


Difference 

-3.406 

(1.003) 

*** 

-5.372 

-1.440 
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Table B.1.18. Model predicting race-moderated Fall Kindergarten student weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.641 

(0.019) 

*** 

Alternative treatment 

0.480 

(0.094) 

*** 

Student is Black 

0.064 

(0.107) 


Student is Asian 

-0.161 

(0.104) 


Student is Hispanic 

-0.162 

(0.150) 


Student is Other 

0.419 

(0.203) 

* 

Program 

-0.397 

(0.095) 

*** 

Time b 

0.215 

(0.016) 

*** 

Program xtime 

0.163 

(0.021) 

*** 

Time squared 

-0.005 

(0.001) 

*** 

Program xtime squared 

-0.007 

(0.001) 

*** 

Program x 




Student is Black 

0.286 

(0.139) 

* 

Student is Asian 

-0.018 

(0.135) 


Student is Hispanic 

0.457 

(0.229) 

* 

Student is Other 

-0.537 

(0.301) 


Time x 




Student is Black 

-0.087 

(0.023) 

*** 

Student is Asian 

-0.006 

(0.022) 


Student is Hispanic 

-0.048 

(0.032) 


Student is Other 

-0.132 

(0.045) 

** 

Program xtime x 




Student is Black 

-0.043 

(0.030) 


Student is Asian 

0.093 

(0.030) 

** 

Student is Hispanic 

-0.055 

(0.050) 


Student is Other 

0.174 

(0.066) 

** 

Time squared x 




Student is Black 

0.004 

(0.001) 

*** 

Student is Asian 

0.001 

(0.001) 


Student is Hispanic 

0.005 

(0.002) 

** 

Student is Other 

0.005 

(0.002) 

* 

Program xtime squared x 




Student is Black 

0.002 

(0.002) 


Student is Asian 

-0.004 

(0.002) 

* 

Student is Hispanic 

0.000 

(0.003) 


Student is Other 

-0.006 

(0.004) 


Intercept 

-0.075 

(0.087) 


Observations 




Weeks 

4474 



Students 

359 



Pairs 

193 



Schools 

17 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

** p < 0.01 
***p< 0.001 

Kindergarten sample includes 107 White, 119 Black, and 97 Asian students 
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Table B.1.19. Model predicting race-moderated Fall first grade students' weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.479 

(0.012) 

*** 

Alternative treatment 

-0.045 

(0.020) 

* 

Student is Black 

-0.075 

(0.029) 

** 

Student is Asian 

-0.056 

(0.028) 

* 

Student is Hispanic 

-0.088 

(0.033) 

** 

Student is Other 

0.015 

(0.122) 


Program 

-0.037 

(0.022) 


Time b 

0.058 

(0.001) 

*** 

Program xtime 

0.005 

(0.002) 

* 

Program x 




Student is Black 

-0.025 

(0.040) 


Student is Asian 

0.035 

(0.034) 


Student is Hispanic 

0.035 

(0.052) 


Student is Other 

-0.026 

(0.175) 


Time x 




Student is Black 

0.002 

(0.002) 


Student is Asian 

0.001 

(0.002) 


Student is Hispanic 

-0.005 

(0.003) 


Student is Other 

-0.016 

(0.011) 


Program xtime x 




Student is Black 

0.010 

(0.004) 

** 

Student is Asian 

0.003 

(0.003) 


Student is Hispanic 

0.008 

(0.005) 


Student is Other 

-0.002 

(0.015) 


Intercept 

1.700 

(0.041) 

*** 

Observations 




Weeks 

5078 



Students 

409 



Pairs 

215 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

** p < 0.01 
***p< 0.001 

First grade sample includes 161 White, 88 Black, and 108 Asian students 
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Table B.1.20. Model predicting race-moderated Fall third grade students' weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.745 

(0.011) 

*** 

Alternative treatment 

-0.038 

(0.015) 

* 

Student is Black 

-0.052 

(0.020) 

** 

Student is Asian 

-0.105 

(0.016) 

*** 

Student is Hispanic 

-0.068 

(0.027) 

* 

Student is Other 

-0.018 

(0.070) 


Program 

-0.057 

(0.014) 

*** 

Time b 

0.025 

(0.001) 

*** 

Program xtime 

0.005 

(0.001) 

*** 

Program x 




Student is Black 

-0.010 

(0.026) 


Student is Asian 

0.032 

(0.021) 


Student is Hispanic 

0.074 

(0.038) 


Student is Other 

-0.138 

(0.077) 


Time x 




Student is Black 

0.000 

(0.002) 


Student is Asian 

0.003 

(0.001) 

* 

Student is Hispanic 

-0.001 

(0.002) 


Student is Other 

-0.004 

(0.006) 


Program xtime x 




Student is Black 

0.000 

(0.002) 


Student is Asian 

-0.004 

(0.002) 

* 

Student is Hispanic 

-0.007 

(0.003) 

* 

Student is Other 

0.014 

(0.006) 

* 

Intercept 

1.110 

(0.045) 

*** 

Observations 




Weeks 

3617 



Students 

305 



Pairs 

167 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p<0.001 

Third grade sample includes 121 White, 71 Black, and 81 Asian students 
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Table B.1.21. Estimated scores for Kindergarten White program participants and control students 


Kindergarten (N=107) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

2.144 

(0.229) 

*** 

1.695 

2.593 

Week 1 

Program 

1.382 

(0.154) 

*** 

1.080 

1.683 


Difference 

-0.762 

(0.232) 

** 

-1.217 

-0.308 


Control 

4.602 

(0.332) 

*** 

3.951 

5.253 

Week 4 

Program 

5.225 

(0.329) 

*** 

4.581 

5.870 


Difference 

0.623 

(0.337) 


-0.038 

1.285 


Control 

8.201 

(0.540) 

*** 

7.142 

9.260 

Week 7 

Program 

12.210 

(0.690) 

*** 

10.858 

13.563 


Difference 

4.010 

(0.635) 

*** 

2.765 

5.255 


Control 

12.928 

(0.822) 

*** 

11.318 

14.539 

Week 10 

Program 

21.757 

(1.196) 

*** 

19.414 

24.101 


Difference 

8.829 

(1.074) 

*** 

6.724 

10.934 


Control 

18.435 

(1.131) 

*** 

16.219 

20.651 

Week 13 

Program 

30.826 

(1.650) 

*** 

27.592 

34.059 


Difference 

12.391 

(1.463) 

*** 

9.523 

15.259 


Control 

23.996 

(1.646) 

*** 

20.770 

27.222 

Week 16 

Program 

35.131 

(2.085) 

*** 

31.045 

39.218 


Difference 

11.135 

(2.085) 

*** 

7.050 

15.221 
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Table B.1.22. Estimated scores for Kindergarten Black program participants and control students 


Kindergarten (N=119) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

2.086 

(0.234) 

*** 

1.627 

2.546 

Week 1 

Program 

1.987 

(0.213) 

*** 

1.568 

2.405 


Difference 

-0.100 

(0.260) 


-0.610 

0.410 


Control 

3.520 

(0.278) 

*** 

2.976 

4.064 

Week 4 

Program 

4.822 

(0.346) 

*** 

4.144 

5.500 


Difference 

1.302 

(0.321) 

*** 

0.673 

1.931 


Control 

5.593 

(0.398) 

*** 

4.812 

6.373 

Week 7 

Program 

9.353 

(0.607) 

*** 

8.163 

10.543 


Difference 

3.760 

(0.533) 

*** 

2.716 

4.804 


Control 

8.574 

(0.580) 

*** 

7.437 

9.711 

Week 10 

Program 

15.794 

(0.986) 

*** 

13.861 

17.726 


Difference 

7.220 

(0.857) 

*** 

5.540 

8.899 


Control 

12.845 

(0.829) 

*** 

11.220 

14.470 

Week 13 

Program 

23.850 

(1.444) 

*** 

21.021 

26.680 


Difference 

11.005 

(1.241) 

*** 

8.574 

13.437 


Control 

18.936 

(1355) 

*** 

16.280 

21.593 

Week 16 

Program 

32.545 

(2.183) 

*** 

28.267 

36.823 


Difference 

13.609 

(2.051) 

*** 

9.588 

17.629 
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Table B.1.23. Estimated scores for Kindergarten Asian program participants and control students 


Kindergarten (N=97) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

1.665 

(0.190) 

*** 

1.292 

2.038 

Week 1 

Program 

1.168 

(0.161) 

*** 

0.852 

1.485 


Difference 

-0.496 

(0.196) 

* 

-0.881 

-0.112 


Control 

3.759 

(0.283) 

*** 

3.204 

4.313 

Week 4 

Program 

6.097 

(0.439) 

*** 

5.236 

6.957 


Difference 

2.338 

(0.366) 

*** 

1.620 

3.056 


Control 

7.020 

(0.474) 

*** 

6.091 

7.950 

Week 7 

Program 

17.047 

(1.110) 

*** 

14.872 

19.221 


Difference 

10.026 

(0.928) 

*** 

8.207 

11.845 


Control 

11.758 

(0.760) 

*** 

10.268 

13.248 

Week 10 

Program 

34.659 

(2.204) 

*** 

30.339 

38.979 


Difference 

22.901 

(1.889) 

*** 

19.200 

26.602 


Control 

18.155 

(1.127) 

*** 

15.945 

20.364 

Week 13 

Program 

53.750 

(3.347) 

*** 

47.190 

60.310 


Difference 

35.595 

(2.859) 

*** 

29.993 

41.198 


Control 

26.142 

(1.765) 

*** 

22.683 

29.601 

Week 16 

Program 

64.317 

(4.460) 

*** 

55.575 

73.059 


Difference 

38.175 

(3.970) 

*** 

30.395 

45.956 
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Table B.1.24. Estimated scores for first grade White program participants and control students 


FirstGrade (N=161) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

25.185 

(0.439) 

*** 

24.325 

26.045 

Week 1 

Program 

24.204 

(0.387) 

*** 

23.445 

24.963 


Difference 

-0.981 

(0.527) 


-2.014 

0.053 


Control 

30.174 

(0.433) 

*** 

29.325 

31.022 

Week 4 

Program 

29.420 

(0.393) 

*** 

28.649 

30.191 


Difference 

-0.754 

(0.501) 


-1.735 

0.227 


Control 

36.113 

(0.447) 

*** 

35.237 

36.988 

Week 7 

Program 

35.714 

(0.419) 

*** 

34.893 

36.535 


Difference 

-0.398 

(0.495) 


-1.368 

0.571 


Control 

43.183 

(0.517) 

*** 

42.170 

44.196 

Week 10 

Program 

43.311 

(0.496) 

*** 

42.340 

44.283 


Difference 

0.128 

(0.573) 


-0.994 

1.250 


Control 

51.601 

(0.684) 

*** 

50.260 

52.941 

Week 13 

Program 

52.480 

(0.659) 

*** 

51.188 

53.772 


Difference 

0.880 

(0.799) 


-0.687 

2.446 


Control 

61.622 

(0.972) 

*** 

59.716 

63.527 

Week 16 

Program 

63.547 

(0.937) 

*** 

61.710 

65.383 


Difference 

1.925 

(1.204) 


-0.434 

4.284 
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Table B.1.25. Estimated scores for first grade Black program participants and control students 


FirstGrade (N=88) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

23.327 

(0.537) 

*** 

22.275 

24.379 

Week 1 

Program 

22.083 

(0.536) 

*** 

21.032 

23.133 


Difference 

-1.245 

(0.723) 


-2.661 

0.172 


Control 

28.122 

(0.531) 

*** 

27.082 

29.162 

Week 4 

Program 

27.897 

(0.551) 

*** 

26.816 

28.978 


Difference 

-0.225 

(0.712) 


-1.620 

1.171 


Control 

33.861 

(0.549) 

*** 

32.784 

34.938 

Week 7 

Program 

35.176 

(0.594) 

*** 

34.011 

36.341 


Difference 

1.315 

(0.734) 


-0.124 

2.754 


Control 

40.732 

(0.641) 

*** 

39.476 

41.987 

Week 10 

Program 

44.289 

(0.726) 

*** 

42.866 

45.711 


Difference 

3.557 

(0.874) 

*** 

1.843 

5.270 


Control 

48.956 

(0.856) 

*** 

47.278 

50.634 

Week 13 

Program 

55.696 

(1.021) 

*** 

53.696 

57.697 


Difference 

6.740 

(1.233) 

*** 

4.324 

9.156 


Control 

58.802 

(1.230) 

*** 

56.392 

61.212 

Week 16 

Program 

69.978 

(1.543) 

*** 

66.954 

73.002 


Difference 

11.176 

(1.874) 

*** 

7.503 

14.849 
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Table B.1.26. Estimated scores for first grade Asian program participants and control students 


FirstGrade (N=108) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

23.789 

(0.504) 

*** 

22.801 

24.777 

Week 1 

Program 

23.797 

(0.442) 

*** 

22.931 

24.663 


Difference 

0.008 

(0.609) 


-1.186 

1.202 


Control 

28.588 

(0.502) 

*** 

27.605 

29.572 

Week 4 

Program 

29.295 

(0.458) 

*** 

28.398 

30.192 


Difference 

0.707 

(0.590) 


-0.449 

1.863 


Control 

34.317 

(0.521) 

*** 

33.295 

35.338 

Week 7 

Program 

36.013 

(0.498) 

*** 

35.036 

36.989 


Difference 

1.696 

(0.596) 

** 

0.527 

2.865 


Control 

41.154 

(0.602) 

*** 

39.973 

42.334 

Week 10 

Program 

44.219 

(0.601) 

*** 

43.042 

45.397 


Difference 

3.066 

(0.697) 

*** 

1.699 

4.432 


Control 

49.315 

(0.791) 

*** 

47.765 

50.865 

Week 13 

Program 

54.246 

(0.809) 

*** 

52.661 

55.831 


Difference 

4.931 

(0.967) 

*** 

3.036 

6.827 


Control 

59.055 

(1.117) 

*** 

56.865 

61.245 

Week 16 

Program 

66.495 

(1.158) 

*** 

64.226 

68.764 


Difference 

7.440 

(1.446) 

*** 

4.606 

10.274 
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Table B.1.27. Estimated scores for third grade White program participants and control students 


Third Grade (N=121) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

58.086 

(0.594) 

*** 

56.921 

59.251 

Week 1 

Program 

55.477 

(0.505) 

*** 

54.486 

56.467 


Difference 

-2.609 

(0.769) 

*** 

-4.117 

-1.101 


Control 

62.750 

(0.532) 

*** 

61.707 

63.793 

Week 4 

Program 

60.877 

(0.464) 

*** 

59.967 

61.787 


Difference 

-1.873 

(0.692) 

** 

-3.230 

-0.517 


Control 

67.783 

(0.494) 

*** 

66.815 

68.751 

Week 7 

Program 

66.794 

(0.443) 

*** 

65.925 

67.662 


Difference 

-0.989 

(0.646) 


-2.255 

0.276 


Control 

73.213 

(0.507) 

*** 

72.219 

74.207 

Week 10 

Program 

73.276 

(0.466) 

*** 

72.362 

74.190 


Difference 

0.063 

(0.669) 


-1.247 

1.373 


Control 

79.072 

(0.595) 

*** 

77.905 

80.239 

Week 13 

Program 

80.378 

(0.554) 

*** 

79.293 

81.464 


Difference 

1.307 

(0.792) 


-0.247 

2.860 


Control 

85.393 

(0.759) 

*** 

83.906 

86.881 

Week 16 

Program 

88.160 

(0.710) 

*** 

86.769 

89.551 


Difference 

2.767 

(1.020) 

** 

0.768 

4.766 
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Table B.1.28. Estimated scores for third grade Black program participants and control students 


Third Grade (N=71) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

55.136 

(0.876) 

*** 

53.418 

56.853 

Week 1 

Program 

52.123 

(0.736) 

*** 

50.680 

53.566 


Difference 

-3.013 

(1.137) 

** 

-5.241 

-0.784 


Control 

59.619 

(0.789) 

*** 

58.072 

61.166 

Week 4 

Program 

57.339 

(0.678) 

*** 

56.011 

58.667 


Difference 

-2.280 

(1.030) 

* 

-4.299 

-0.261 


Control 

64.461 

(0.739) 

*** 

63.013 

65.909 

Week 7 

Program 

63.067 

(0.651) 

*** 

61.791 

64.344 


Difference 

-1.394 

(0.972) 


-3.299 

0.512 


Control 

69.689 

(0.766) 

*** 

68.188 

71.191 

Week 10 

Program 

69.358 

(0.694) 

*** 

67.998 

70.717 


Difference 

-0.331 

(1.019) 


-2.329 

1.666 


Control 

75.335 

(0.902) 

*** 

73.566 

77.104 

Week 13 

Program 

76.266 

(0.834) 

*** 

74.631 

77.901 


Difference 

0.931 

(1.214) 


-1.450 

3.311 


Control 

81.432 

(1.149) 

*** 

79.180 

83.684 

Week 16 

Program 

83.853 

(1.078) 

*** 

81.740 

85.966 


Difference 

2.420 

(1.562) 


-0.641 

5.482 
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Table B.1.29. Estimated scores for third grade Asian program participants and control students 


Third Grade (N=81) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

52.346 

(0.585) 

*** 

51.200 

53.491 

Week 1 

Program 

51.445 

(0.570) 

*** 

50.328 

52.562 


Difference 

-0.900 

(0.793) 


-2.455 

0.654 


Control 

57.072 

(0.544) 

*** 

56.007 

58.138 

Week 4 

Program 

56.351 

(0.530) 

*** 

55.312 

57.391 


Difference 

-0.721 

(0.729) 


-2.151 

0.709 


Control 

62.218 

(0.531) 

*** 

61.176 

63.259 

Week 7 

Program 

61.716 

(0.517) 

*** 

60.704 

62.728 


Difference 

-0.502 

(0.704) 


-1.882 

0.878 


Control 

67.819 

(0.572) 

*** 

66.698 

68.940 

Week 10 

Program 

67.583 

(0.553) 

*** 

66.498 

68.667 


Difference 

-0.236 

(0.755) 


-1.716 

1.243 


Control 

73.917 

(0.683) 

*** 

72.579 

75.254 

Week 13 

Program 

73.998 

(0.660) 

*** 

72.706 

75.291 


Difference 

0.082 

(0.908) 


-1.698 

1.862 


Control 

80.555 

(0.864) 

*** 

78.860 

82.249 

Week 16 

Program 

81.014 

(0.839) 

*** 

79.371 

82.658 


Difference 

0.459 

(1.166) 


-1.825 

2.744 
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Table B.1.30. Model predicting race-moderated (non-White) Fall Kindergarten students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.634 

(0.020) 

*** 

Alternative treatment 

0.443 

(0.096) 

*** 

Non-White 

-0.096 

(0.091) 


Program 

-0.417 

(0.098) 

*** 

Time b 

0.214 

(0.016) 

*** 

Program xtime 

0.164 

(0.022) 

*** 

Time squared 

-0.004 

(0.001) 

*** 

Program xtime squared 

-0.007 

(0.001) 

*** 

Non-White x program 

0.161 

(0.118) 


Non-White xtime 

-0.048 

(0.019) 

* 

Non-White x program xtime 

0.019 

(0.026) 


Non-White xtime squared 

0.003 

(0.001) 

** 

Non-White x program xtime squared 

-0.001 

(0.001) 


Intercept 

-0.031 

(0.089) 


Observations 




Weeks 

4474 



Students 

359 



Pairs 

193 



Schools 

17 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
*** p < 0.001 

Kindergarten sample includes 107 White and 252 non-White students 
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Table B.1.31. Model predicting race-moderated (non-White) Fall first grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.477 

(0.011) 

*** 

Alternative treatment 

-0.059 

(0.020) 

** 

Non-White 

-0.075 

(0.022) 

*** 

Program 

-0.039 

(0.022) 


Time b 

0.058 

(0.001) 

*** 

Program xtime 

0.005 

(0.002) 

* 

Non-White x program 

0.021 

(0.029) 


Non-White xtime 

0.000 

(0.002) 


Non-White x program xtime 

0.006 

(0.003) 

* 

Intercept 

1.712 

(0.041) 

*** 

Observations 




Weeks 

5078 



Students 

409 



Pairs 

215 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
* p <0.05 

**p <0.01 
***p <0.001 

Firstgrade sample includes 161 White and 248 non-White students 
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Table B.1.32. Model predicting race-moderated (non-White) Fall third grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.682 

(0.017) 

*** 

Alternative treatment 

-0.042 

(0.027) 


Non-White 

-0.008 

(0.027) 


Program 

-0.041 

(0.028) 


Time b 

0.036 

(0.001) 

*** 

Program xtime 

0.008 

(0.002) 

*** 

Non-White x program 

-0.004 

(0.035) 


Non-White xtime 

-0.005 

(0.002) 

* 

Non-White x program xtime 

-0.005 

(0.003) 


Intercept 

1.132 

(0.064) 

*** 

Observations 




Weeks 

3617 



Students 

305 



Pairs 

167 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
*** p <0.001 

Third grade sample includes 121 White and 184 non-White students 
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Table B.1.33. Estimated scores for Kindergarten White program participants and control students 


Kindergarten (N=107) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

2.144 

(0.229) 

*** 

1.695 

2.593 

Week 1 

Program 

1.382 

(0.154) 

*** 

1.080 

1.683 


Difference 

-0.762 

(0.232) 

** 

-1.217 

-0.308 


Control 

4.602 

(0.332) 

*** 

3.951 

5.253 

Week 4 

Program 

5.225 

(0.329) 

*** 

4.581 

5.870 


Difference 

0.623 

(0.337) 


-0.038 

1.285 


Control 

8.201 

(0.540) 

*** 

7.142 

9.260 

Week 7 

Program 

12.210 

(0.690) 

*** 

10.858 

13.563 


Difference 

4.010 

(0.635) 

*** 

2.765 

5.255 


Control 

12.928 

(0.822) 

*** 

11.318 

14.539 

Week 10 

Program 

21.757 

(1.196) 

*** 

19.414 

24.101 


Difference 

8.829 

(1.074) 

*** 

6.724 

10.934 


Control 

18.435 

(1.131) 

*** 

16.219 

20.651 

Week 13 

Program 

30.826 

(1.650) 

*** 

27.592 

34.059 


Difference 

12.391 

(1.463) 

*** 

9.523 

15.259 


Control 

23.996 

(1.646) 

*** 

20.770 

27.222 

Week 16 

Program 

35.131 

(2.085) 

*** 

31.045 

39.218 


Difference 

11.135 

(2.085) 

*** 

7.050 

15.221 
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Table B.1.34. Estimated scores for Kindergarten non-White program participants and control students 


Kindergarten (N=252) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

1.805 

(0.161) 

*** 

1.490 

2.121 

Week 1 

Program 

1.500 

(0.146) 

*** 

1.214 

1.787 


Difference 

-0.305 

(0.148) 

* 

-0.594 

-0.016 


Control 

3.524 

(0.228) 

*** 

3.078 

3.970 

Week 4 

Program 

5.206 

(0.320) 

*** 

4.579 

5.832 


Difference 

1.682 

(0.228) 

*** 

1.236 

2.128 


Control 

6.111 

(0.356) 

*** 

5.413 

6.808 

Week 7 

Program 

12.030 

(0.667) 

*** 

10.723 

13.338 


Difference 

5.920 

(0.484) 

*** 

4.971 

6.868 


Control 

9.896 

(0.547) 

*** 

8.823 

10.969 

Week 10 

Program 

22.149 

(1.188) 

*** 

19.820 

24.477 


Difference 

12.252 

(0.889) 

*** 

10.509 

13.996 


Control 

15.276 

(0.812) 

*** 

13.686 

16.867 

Week 13 

Program 

33.793 

(1.772) 

*** 

30.319 

37.266 


Difference 

18.516 

(1.317) 

*** 

15.935 

21.097 


Control 

22.701 

(1.270) 

*** 

20.211 

25.191 

Week 16 

Program 

43.243 

(2.424) 

*** 

38.493 

47.994 


Difference 

20.542 

(1.916) 

*** 

16.787 

24.297 
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Table B.1.35. Estimated scores for first grade White program participants and control students 


FirstGrade (N=161) 



Control 

25.185 

(0.439) 

*** 

24.325 

26.045 

Week 1 

Program 

24.204 

(0.387) 

*** 

23.445 

24.963 


Difference 

-0.981 

(0.527) 


-2.014 

0.053 









Control 

36.113 

(0.447) 

*** 

35.237 

36.988 

Week 7 

Program 

35.714 

(0.419) 

*** 

34.893 

36.535 


Difference 

-0.398 

(0.495) 


-1.368 

0.571 


Control 

43.183 

(0.517) 

*** 

42.170 

44.196 

Week 10 

Program 

43.311 

(0.496) 

*** 

42.340 

44.283 


Difference 

0.128 

(0.573) 


-0.994 

1.250 


Control 

51.601 

(0.684) 

*** 

50.260 

52.941 

Week 13 

Program 

52.480 

(0.659) 

*** 

51.188 

53.772 


Difference 

0.880 

(0.799) 


-0.687 

2.446 


Control 

61.622 

(0.972) 

*** 

59.716 

63.527 

Week 16 

Program 

63.547 

(0.937) 

*** 

61.710 

65.383 


Difference 

1.925 

(1.204) 


-0.434 

4.284 
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Table B.1.36. Estimated scores for first grade non-White program participants and control students 


FirstGrade (N=248) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

23.348 

(0.332) 

*** 

22.698 

23.998 

Week 1 

Program 

23.080 

(0.331) 

*** 

22.432 

23.728 


Difference 

-0.268 

(0.414) 


-1.080 

0.544 


Control 

27.956 

(0.333) 

*** 

27.303 

28.609 

Week 4 

Program 

28.601 

(0.345) 

*** 

27.924 

29.278 


Difference 

0.645 

(0.401) 


-0.141 

1.432 


Control 

33.435 

(0.350) 

*** 

32.749 

34.121 

Week 7 

Program 

35.387 

(0.378) 

*** 

34.646 

36.128 


Difference 

1.952 

(0.406) 

*** 

1.156 

2.748 


Control 

39.951 

(0.406) 

*** 

39.156 

40.746 

Week 10 

Program 

43.729 

(0.457) 

*** 

42.833 

44.625 


Difference 

3.778 

(0.476) 

*** 

2.844 

4.712 


Control 

47.701 

(0.527) 

*** 

46.668 

48.733 

Week 13 

Program 

53.984 

(0.615) 

*** 

52.778 

55.189 


Difference 

6.283 

(0.665) 

*** 

4.979 

7.587 


Control 

56.916 

(0.733) 

*** 

55.480 

58.352 

Week 16 

Program 

66.589 

(0.882) 

*** 

64.861 

68.318 


Difference 

9.673 

(1.002) 

*** 

7.709 

11.637 
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Table B.1.37. Estimated scores for third grade White program participants and control students 


Third Grade (N=121) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

58.086 

(0.594) 

*** 

56.921 

59.251 

Week 1 

Program 

55.477 

(0.505) 

*** 

54.486 

56.467 


Difference 

-2.609 

(0.769) 

*** 

-4.117 

-1.101 


Control 

62.750 

(0.532) 

*** 

61.707 

63.793 

Week 4 

Program 

60.877 

(0.464) 

*** 

59.967 

61.787 


Difference 

-1.873 

(0.692) 

** 

-3.230 

-0.517 


Control 

67.783 

(0.494) 

*** 

66.815 

68.751 

Week 7 

Program 

66.794 

(0.443) 

*** 

65.925 

67.662 


Difference 

-0.989 

(0.646) 


-2.255 

0.276 


Control 

73.213 

(0.507) 

*** 

72.219 

74.207 

Week 10 

Program 

73.276 

(0.466) 

*** 

72.362 

74.190 


Difference 

0.063 

(0.669) 


-1.247 

1.373 


Control 

79.072 

(0.595) 

*** 

77.905 

80.239 

Week 13 

Program 

80.378 

(0.554) 

*** 

79.293 

81.464 


Difference 

1.307 

(0.792) 


-0.247 

2.860 


Control 

85.393 

(0.759) 

*** 

83.906 

86.881 

Week 16 

Program 

88.160 

(0.710) 

*** 

86.769 

89.551 


Difference 

2.767 

(1.020) 

** 

0.768 

4.766 
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Table B.1.38. Estimated scores for third grade non-White program participants and control students 


Third Grade (N=184) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

53.434 

(0.454) 

*** 

52.544 

54.324 

Week 1 

Program 

51.863 

(0.412) 

*** 

51.055 

52.671 


Difference 

-1.571 

(0.598) 

** 

-2.743 

-0.399 


Control 

58.021 

(0.418) 

*** 

57.203 

58.840 

Week 4 

Program 

56.854 

(0.382) 

*** 

56.105 

57.603 


Difference 

-1.167 

(0.546) 

* 

-2.237 

-0.097 


Control 

62.995 

(0.402) 

*** 

62.207 

63.783 

Week 7 

Program 

62.316 

(0.370) 

*** 

61.591 

63.041 


Difference 

-0.679 

(0.522) 


-1.702 

0.344 


Control 

68.388 

(0.427) 

*** 

67.550 

69.225 

Week 10 

Program 

68.294 

(0.395) 

*** 

67.520 

69.067 


Difference 

-0.094 

(0.554) 


-1.181 

0.992 


Control 

74.235 

(0.506) 

*** 

73.243 

75.228 

Week 13 

Program 

74.836 

(0.471) 

*** 

73.913 

75.758 


Difference 

0.600 

(0.664) 


-0.701 

1.902 


Control 

80.575 

(0.641) 

*** 

79.320 

81.831 

Week 16 

Program 

81.995 

(0.601) 

*** 

80.818 

83.172 


Difference 

1.420 

(0.853) 


-0.252 

3.092 
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Table B.1.39. Model predicting Dual Language Learner -moderated Fall Kindergarten student weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.642 

(0.020) 

*** 

Alternative treatment 

0.431 

(0.096) 

*** 

Dual Language Learner 

-0.214 

(0.085) 

* 

Program 

-0.286 

(0.065) 

*** 

Time b 

0.179 

(0.010) 

*** 

Program xtime 

0.155 

(0.014) 

*** 

Time squared 

-0.003 

(0.001) 

*** 

Program xtime squared 

-0.007 

(0.001) 

*** 

Dual Language Learner x program 

-0.063 

(0.117) 


Dual Language Learner x time 

0.004 

(0.018) 


Dual Language Learner x program xtime 

0.084 

(0.026) 

** 

Dual Language Learner x time squared 

0.001 

(0.001) 


Dual Language Learner x program x time squared 

-0.004 

(0.001) 

* 

Intercept 

-0.027 

(0.067) 


Observations 




Weeks 

4474 



Students 

359 



Pairs 

193 



Schools 

17 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
*** p < 0.001 

Kindergarten sample includes 264 non-Dual Language Learners and 95 Dual Language Learners 
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Table B.1.40. Model predicting Dual Language Learner -moderated Fall first grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.480 

(0.011) 

*** 

Alternative treatment 

-0.058 

(0.020) 

** 

Dual Language Learner 

-0.040 

(0.022) 


Program 

-0.037 

(0.018) 

* 

Time b 

0.058 

(0.001) 

*** 

Program xtime 

0.007 

(0.002) 

*** 

Dual Language Learner x program 

0.035 

(0.030) 


Dual Language Learner x time 

0.000 

(0.002) 


Dual Language Learner x program xtime 

0.003 

(0.003) 


Intercept 

1.669 

(0.039) 

*** 

Observations 




Weeks 

5078 



Students 

409 



Pairs 

215 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p< 0.001 

Firstgrade sample includes 263 non-Dual Language Learners and 146 Dual Language Learners 
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Table B.1.41. Model predicting Dual Language Learner-moderated Fall third grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.689 

(0.016) 

*** 

Alternative treatment 

-0.054 

(0.026) 

* 

Dual Language Learner 

-0.097 

(0.027) 

*** 

Program 

-0.076 

(0.022) 

*** 

Time b 

0.033 

(0.001) 

*** 

Program xtime 

0.010 

(0.002) 

*** 

Dual Language Learner x program 

0.084 

(0.036) 

* 

Dual Language Learner x time 

0.000 

(0.002) 


Dual Language Learner x program xtime 

-0.013 

(0.003) 

*** 

Intercept 

1.140 

(0.058) 

*** 

Observations 




Weeks 

3214 



Students 

268 



Pairs 

156 



Schools 

20 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
*** p <0.001 

Third grade sample includes 213 non-Dual Language Learners and 95 Dual Language Learners 
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Table B.1.42. Estimated scores for Kindergarten Non-Dual Language Learner program participants and control 
students 


Kindergarten (N=264) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

2.176 

(0.176) 

*** 

1.832 

2.520 

Week 1 

Program 

1.675 

(0.143) 

*** 

1.395 

1.955 


Difference 

-0.501 

(0.162) 

** 

-0.818 

-0.184 


Control 

4.215 

(0.249) 

*** 

3.726 

4.704 

Week 4 

Program 

5.325 

(0.294) 

*** 

4.748 

5.902 


Difference 

1.109 

(0.227) 

*** 

0.665 

1.554 


Control 

7.154 

(0.388) 

*** 

6.394 

7.914 

Week 7 

Program 

11.626 

(0.583) 

*** 

10.483 

12.768 


Difference 

4.472 

(0.431) 

*** 

3.628 

5.316 


Control 

11.140 

(0.579) 

*** 

10.004 

12.275 

Week 10 

Program 

20.282 

(0.985) 

*** 

18.351 

22.213 


Difference 

9.142 

(0.736) 

*** 

7.700 

10.584 


Control 

16.209 

(0.815) 

*** 

14.613 

17.806 

Week 13 

Program 

29.290 

(1.390) 

*** 

26.565 

32.015 


Difference 

13.081 

(1.026) 

*** 

11.070 

15.091 


Control 

22.229 

(1.196) 

*** 

19.886 

24.573 

Week 16 

Program 

35.401 

(1.798) 

*** 

31.877 

38.926 


Difference 

13.172 

(1.469) 

*** 

10.292 

16.052 
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Table B.1.43. Estimated scores for Kindergarten Dual Language Learner program participants and control students 


Kindergarten (N=95) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

1.577 

(0.185) 

*** 

1.214 

1.939 

Week 1 

Program 

1.209 

(0.163) 

*** 

0.889 

1.528 


Difference 

-0.368 

(0.198) 


-0.757 

0.020 


Control 

3.361 

(0.257) 

*** 

2.857 

3.864 

Week 4 

Program 

5.562 

(0.404) 

*** 

4.770 

6.353 


Difference 

2.201 

(0.347) 

*** 

1.521 

2.880 


Control 

6.192 

(0.420) 

*** 

5.369 

7.015 

Week 7 

Program 

14.806 

(0.968) 

*** 

12.909 

16.702 


Difference 

8.614 

(0.825) 

*** 

6.997 

10.230 


Control 

10.561 

(0.679) 

*** 

9.231 

11.891 

Week 10 

Program 

29.870 

(1.900) 

*** 

26.145 

33.595 


Difference 

19.309 

(1.648) 

*** 

16.079 

22.539 


Control 

17.112 

(1.049) 

*** 

15.056 

19.169 

Week 13 

Program 

47.884 

(2.978) 

*** 

42.048 

53.720 


Difference 

30.772 

(2.570) 

*** 

25.734 

35.809 


Control 

26.656 

(1.787) 

*** 

23.154 

30.158 

Week 16 

Program 

61.763 

(4.292) 

*** 

53.350 

70.176 


Difference 

35.107 

(3.877) 

*** 

27.508 

42.706 
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Table B.1.44. Estimated scores for first grade Non-Dual Language Learner program participants and control 
students 


FirstGrade (N=263) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

24.457 

(0.334) 

*** 

23.803 

25.111 

Week 1 

Program 

23.606 

(0.319) 

*** 

22.981 

24.232 


Difference 

-0.851 

(0.418) 

* 

-1.671 

-0.032 


Control 

29.292 

(0.331) 

*** 

28.643 

29.941 

Week 4 

Program 

28.903 

(0.326) 

*** 

28.265 

29.542 


Difference 

-0.389 

(0.401) 


-1.175 

0.397 


Control 

35.045 

(0.344) 

*** 

34.371 

35.719 

Week 7 

Program 

35.341 

(0.349) 

*** 

34.657 

36.025 


Difference 

0.296 

(0.401) 


-0.491 

1.083 


Control 

41.890 

(0.399) 

*** 

41.109 

42.672 

Week 10 

Program 

43.164 

(0.415) 

*** 

42.350 

43.979 


Difference 

1.274 

(0.467) 

** 

0.359 

2.189 


Control 

50.036 

(0.524) 

*** 

49.008 

51.064 

Week 13 

Program 

52.672 

(0.557) 

*** 

51.580 

53.764 


Difference 

2.636 

(0.650) 

*** 

1.363 

3.909 


Control 

59.728 

(0.740) 

*** 

58.278 

61.179 

Week 16 

Program 

64.227 

(0.798) 

*** 

62.662 

65.792 


Difference 

4.498 

(0.975) 

*** 

2.588 

6.409 
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Table B.1.45. Estimated scores for first grade Dual Language Learner program participants and control students 


FirstGrade (N=146) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

23.459 

(0.424) 

*** 

22.629 

24.289 

Week 1 

Program 

23.572 

(0.399) 

*** 

22.791 

24.354 


Difference 

0.113 

(0.533) 


-0.932 

1.158 


Control 

28.093 

(0.420) 

*** 

27.271 

28.916 

Week 4 

Program 

29.136 

(0.413) 

*** 

28.327 

29.946 


Difference 

1.043 

(0.518) 

* 

0.028 

2.058 


Control 

33.605 

(0.435) 

*** 

32.754 

34.457 

Week 7 

Program 

35.961 

(0.449) 

*** 

35.080 

36.841 


Difference 

2.355 

(0.526) 

*** 

1.324 

3.386 


Control 

40.162 

(0.502) 

*** 

39.178 

41.146 

Week 10 

Program 

44.330 

(0.542) 

*** 

43.267 

45.392 


Difference 

4.168 

(0.616) 

*** 

2.960 

5.376 


Control 

47.961 

(0.659) 

*** 

46.669 

49.252 

Week 13 

Program 

54.594 

(0.733) 

*** 

53.158 

56.031 


Difference 

6.634 

(0.854) 

*** 

4.960 

8.308 


Control 

57.237 

(0.930) 

*** 

55.414 

59.059 

Week 16 

Program 

67.183 

(1.056) 

*** 

65.113 

69.253 


Difference 

9.946 

(1.276) 

*** 

7.445 

12.447 
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Table B.1.46. Estimated scores for third grade Non-Dual Language Learner program participants and control 
students 


Third Grade (N=213) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

56.994 

(0.485) 

*** 

56.043 

57.945 

Week 1 

Program 

54.164 

(0.403) 

*** 

53.375 

54.953 


Difference 

-2.830 

(0.605) 

*** 

-4.015 

-1.644 


Control 

61.555 

(0.441) 

*** 

60.692 

62.419 

Week 4 

Program 

59.524 

(0.376) 

*** 

58.788 

60.261 


Difference 

-2.031 

(0.546) 

*** 

-3.100 

-0.961 


Control 

66.475 

(0.416) 

*** 

65.661 

67.290 

Week 7 

Program 

65.405 

(0.366) 

*** 

64.689 

66.122 


Difference 

-1.070 

(0.511) 

* 

-2.072 

-0.068 


Control 

71.782 

(0.430) 

*** 

70.939 

72.626 

Week 10 

Program 

71.857 

(0.389) 

*** 

71.095 

72.620 


Difference 

0.075 

(0.532) 


-0.967 

1.118 


Control 

77.507 

(0.501) 

*** 

76.524 

78.489 

Week 13 

Program 

78.937 

(0.461) 

*** 

78.034 

79.840 


Difference 

1.430 

(0.632) 

* 

0.192 

2.668 


Control 

83.681 

(0.630) 

*** 

82.447 

84.915 

Week 16 

Program 

86.704 

(0.584) 

*** 

85.559 

87.848 


Difference 

3.022 

(0.813) 

*** 

1.429 

4.616 
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Table B.1.47. Estimated scores for third grade Dual Language Learner program participants and control students 


Third Grade (N=95) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

52.763 

(0.579) 

*** 

51.628 

53.897 

Week 1 

Program 

51.671 

(0.578) 

*** 

50.538 

52.804 


Difference 

-1.092 

(0.769) 


-2.599 

0.416 


Control 

57.421 

(0.544) 

*** 

56.355 

58.487 

Week 4 

Program 

56.458 

(0.544) 

*** 

55.392 

57.524 


Difference 

-0.963 

(0.707) 


-2.348 

0.423 


Control 

62.482 

(0.536) 

*** 

61.432 

63.533 

Week 7 

Program 

61.680 

(0.534) 

*** 

60.633 

62.727 


Difference 

-0.802 

(0.681) 


-2.137 

0.533 


Control 

67.983 

(0.576) 

*** 

66.853 

69.112 

Week 10 

Program 

67.377 

(0.570) 

*** 

66.260 

68.494 


Difference 

-0.606 

(0.727) 


-2.030 

0.819 


Control 

73.959 

(0.679) 

*** 

72.628 

75.291 

Week 13 

Program 

73.591 

(0.668) 

*** 

72.282 

74.900 


Difference 

-0.368 

(0.869) 


-2.071 

1.334 


Control 

80.454 

(0.848) 

*** 

78.792 

82.115 

Week 16 

Program 

80.370 

(0.833) 

*** 

78.739 

82.002 


Difference 

-0.083 

(1.109) 


-2.257 

2.090 
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Table B.1.48. Model predicting FRPL-moderated Fall Kindergarten grade students' weekly assessments 3 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.629 

(0.020) 

*** 

Alternative treatment 

0.385 

(0.095) 

*** 

FRPL 

-0.034 

(0.109) 


Program 

-0.264 

(0.117) 

* 

Time b 

0.233 

(0.021) 

*** 

Program xtime 

0.152 

(0.026) 

*** 

Time squared 

-0.004 

(0.001) 

*** 

Program xtime squared 

-0.008 

(0.001) 

*** 

FRPL x program 

-0.076 

(0.133) 


FRPL xtime 

-0.065 

(0.023) 

** 

FRPL x program xtime 

0.030 

(0.029) 


FRPL xtime squared 

0.003 

(0.001) 

* 

FRPL x program xtime squared 

0.000 

(0.002) 


Intercept 

-0.079 

(0.110) 


Observations 




Weeks 

4410 



Students 

354 



Pairs 

192 



Schools 

17 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
*** p < 0.001 

Kindergarten sample includes 85 non-FRPL and 269 FRPL students 
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Table B.1.49. Model predicting FRPL-moderated Fall firstgrade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.476 

(0.011) 

*** 

Alternative treatment 

-0.065 

(0.020) 

** 

FRPL 

-0.100 

(0.024) 

*** 

Program 

-0.083 

(0.027) 

** 

Time b 

0.057 

(0.002) 

*** 

Program xtime 

0.006 

(0.002) 

** 

FRPL x program 

0.080 

(0.032) 

* 

FRPL xtime 

0.002 

(0.002) 


FRPL x program xtime 

0.003 

(0.003) 


Intercept 

1.739 

(0.042) 

*** 

Observations 




Weeks 

5000 



Students 

403 



Pairs 

214 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p< 0.001 

Firstgrade sample includes 112 non-FRPL and 291 FRPL students 
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Table B.1.50. Model predicting FRPL-moderated Fall third grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Benchmark 

0.690 

(0.017) 

*** 

Alternative treatment 

-0.057 

(0.026) 

* 

FRPL 

-0.008 

(0.028) 


Program 

-0.055 

(0.032) 


Time b 

0.031 

(0.002) 

*** 

Program xtime 

0.009 

(0.002) 

*** 

FRPL x program 

0.011 

(0.038) 


FRPL xtime 

0.003 

(0.002) 


FRPL x program xtime 

-0.005 

(0.003) 


Intercept 

1.116 

(0.065) 

*** 

Observations 




Weeks 

3556 



Students 

301 



Pairs 

167 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
*** p <0.001 

Third grade sample includes 87 non-FRPL and 217 FRPL students 
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Table B.1.51. Estimated scores for Kindergarten FRPL program participants and control students 


Kindergarten (N=269) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

1.779 

(0.152) 

*** 

1.481 

2.077 

Week 1 

Program 

1.305 

(0.133) 

*** 

1.045 

1.566 


Difference 

-0.474 

(0.136) 

*** 

-0.741 

-0.206 


Control 

3.484 

(0.217) 

*** 

3.059 

3.909 

Week 4 

Program 

4.740 

(0.290) 

*** 

4.171 

5.308 


Difference 

1.256 

(0.205) 

*** 

0.853 

1.659 


Control 

6.002 

(0.337) 

*** 

5.341 

6.664 

Week 7 

Program 

11.123 

(0.608) 

*** 

9.931 

12.315 


Difference 

5.120 

(0.436) 

*** 

4.266 

5.975 


Control 

9.585 

(0.512) 

*** 

8.582 

10.588 

Week 10 

Program 

20.717 

(1.092) 

*** 

18.577 

22.858 


Difference 

11.132 

(0.812) 

*** 

9.540 

12.724 


Control 

14.489 

(0.744) 

*** 

13.032 

15.947 

Week 13 

Program 

32.000 

(1.647) 

*** 

28.773 

35.228 


Difference 

17.511 

(1.223) 

*** 

15.113 

19.909 


Control 

20.939 

(1.130) 

*** 

18.724 

23.154 

Week 16 

Program 

41.535 

(2.282) 

*** 

37.062 

46.008 


Difference 

20.596 

(1.805) 

*** 

17.058 

24.134 
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Table B.1.52. Estimated scores for Kindergarten non-FRPL program participants and control students 


Kindergarten (N=85) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

2.059 

(0.280) 

*** 

1.509 

2.609 

Week 1 

Program 

1.656 

(0.179) 

*** 

1.306 

2.006 


Difference 

-0.403 

(0.291) 


-0.973 

0.167 


Control 

4.763 

(0.401) 

*** 

3.976 

5.550 

Week 4 

Program 

6.019 

(0.389) 

*** 

5.256 

6.782 


Difference 

1.256 

(0.429) 

** 

0.415 

2.097 


Control 

9.028 

(0.680) 

*** 

7.695 

10.361 

Week 7 

Program 

13.900 

(0.820) 

*** 

12.293 

15.508 


Difference 

4.872 

(0.814) 

*** 

3.277 

6.467 


Control 

15.117 

(1.100) 

*** 

12.962 

17.273 

Week 10 

Program 

24.412 

(1.406) 

*** 

21.657 

27.168 


Difference 

9.295 

(1.375) 

*** 

6.600 

11.989 


Control 

22.926 

(1.601) 

*** 

19.788 

26.065 

Week 13 

Program 

33.818 

(1.901) 

*** 

30.093 

37.543 


Difference 

10.892 

(1.881) 

*** 

7.205 

14.580 


Control 

31.805 

(2.538) 

*** 

26.830 

36.781 

Week 16 

Program 

37.326 

(2.343) 

*** 

32.733 

41.919 


Difference 

5.520 

(2.819) 


-0.005 

11.046 
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Table B.1.53. Estimated scores forfirstgrade FRPL program participants and control students 


FirstGrade (N=291) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

23.376 

(0.306) 

*** 

22.776 

23.976 

Week 1 

Program 

23.466 

(0.306) 

*** 

22.865 

24.066 


Difference 

0.090 

(0.387) 


-0.669 

0.849 


Control 

28.042 

(0.307) 

*** 

27.439 

28.644 

Week 4 

Program 

28.969 

(0.317) 

*** 

28.347 

29.591 


Difference 

0.927 

(0.375) 

* 

0.192 

1.662 


Control 

33.601 

(0.323) 

*** 

32.967 

34.234 

Week 7 

Program 

35.709 

(0.345) 

*** 

35.034 

36.385 


Difference 

2.109 

(0.379) 

*** 

1.365 

2.852 


Control 

40.224 

(0.376) 

*** 

39.486 

40.961 

Week 10 

Program 

43.966 

(0.415) 

*** 

43.152 

44.780 


Difference 

3.743 

(0.445) 

*** 

2.870 

4.615 


Control 

48.114 

(0.492) 

*** 

47.151 

49.078 

Week 13 

Program 

54.080 

(0.559) 

*** 

52.984 

55.176 


Difference 

5.966 

(0.622) 

*** 

4.747 

7.184 


Control 

57.516 

(0.687) 

*** 

56.168 

58.863 

Week 16 

Program 

66.469 

(0.803) 

*** 

64.894 

68.044 


Difference 

8.953 

(0.935) 

*** 

7.122 

10.785 
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Table B.1.54. Estimated scores forfirstgrade non-FRPL program participants and control students 


FirstGrade (N=112) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

25.884 

(0.520) 

*** 

24.865 

26.904 

Week 1 

Program 

23.822 

(0.430) 

*** 

22.979 

24.665 


Difference 

-2.062 

(0.641) 

** 

-3.319 

-0.805 


Control 

30.870 

(0.501) 

*** 

29.888 

31.853 

Week 4 

Program 

28.974 

(0.430) 

*** 

28.132 

29.817 


Difference 

-1.896 

(0.611) 

** 

-3.094 

-0.697 


Control 

36.781 

(0.503) 

*** 

35.795 

37.767 

Week 7 

Program 

35.196 

(0.451) 

*** 

34.312 

36.080 


Difference 

-1.585 

(0.607) 

** 

-2.774 

-0.396 


Control 

43.788 

(0.573) 

*** 

42.664 

44.911 

Week 10 

Program 

42.709 

(0.533) 

*** 

41.666 

43.753 


Difference 

-1.078 

(0.698) 


-2.446 

0.290 


Control 

52.094 

(0.765) 

*** 

50.595 

53.592 

Week 13 

Program 

51.782 

(0.719) 

*** 

50.373 

53.191 


Difference 

-0.312 

(0.961) 


-2.195 

1.572 


Control 

61.940 

(1.104) 

*** 

59.777 

64.104 

Week 16 

Program 

62.738 

(1.040) 

*** 

60.699 

64.777 


Difference 

0.798 

(1.431) 


-2.007 

3.602 
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Table B.1.55. Estimated scores forthird grade FRPL program participants and control students 


Third Grade (N=214) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

54.797 

( 0 . 458 ) 

*** 

53.899 

55.695 

Week 1 

Program 

52.848 

( 0 . 415 ) 

*** 

52.035 

53.662 


Difference 

- 1.949 

( 0 . 574 ) 

*** 

- 3.074 

- 0.823 


Control 

59.425 

( 0 . 428 ) 

*** 

58.586 

60.264 

Week 4 

Program 

58.003 

( 0 . 393 ) 

*** 

57.232 

58.774 


Difference 

- 1.422 

( 0 . 525 ) 

** 

- 2.451 

- 0.393 


Control 

64.437 

( 0 . 418 ) 

*** 

63.618 

65.256 

Week 7 

Program 

63.652 

( 0 . 388 ) 

*** 

62.890 

64.413 


Difference 

- 0.785 

( 0 . 502 ) 


- 1.769 

0.198 


Control 

69.865 

( 0 . 443 ) 

*** 

68.995 

70.734 

Week 10 

Program 

69.841 

( 0 . 416 ) 

*** 

69.025 

70.656 


Difference 

- 0.024 

( 0 . 531 ) 


- 1.066 

1.018 


Control 

75.742 

( 0 . 517 ) 

*** 

74.729 

76.756 

Week 13 

Program 

76.622 

( 0 . 488 ) 

*** 

75.666 

77.579 


Difference 

0.880 

( 0 . 634 ) 


- 0.363 

2.123 


Control 

82.108 

( 0 . 641 ) 

*** 

80.852 

83.364 

Week 16 

Program 

84.053 

( 0 . 609 ) 

*** 

82.859 

85.247 


Difference 

1.945 

( 0 . 812 ) 

* 

0.354 

3.537 
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Table B.1.56. Estimated scores for third grade non-FRPL program participants and control students 


Third Grade (N=87) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

56.887 

(0.703) 

*** 

55.509 

58.265 

Week 1 

Program 

55.053 

(0.601) 

*** 

53.876 

56.231 


Difference 

-1.834 

(0.893) 

* 

-3.585 

-0.083 


Control 

61.516 

(0.636) 

*** 

60.269 

62.762 

Week 4 

Program 

60.299 

(0.560) 

*** 

59.200 

61.397 


Difference 

-1.217 

(0.807) 


-2.798 

0.364 


Control 

66.514 

(0.597) 

*** 

65.345 

67.684 

Week 7 

Program 

66.035 

(0.544) 

*** 

64.970 

67.101 


Difference 

-0.479 

(0.755) 


-1.960 

1.001 


Control 

71.913 

(0.616) 

*** 

70.706 

73.120 

Week 10 

Program 

72.308 

(0.574) 

*** 

71.183 

73.434 


Difference 

0.395 

(0.783) 


-1.139 

1.930 


Control 

77.743 

(0.719) 

*** 

76.335 

79.152 

Week 13 

Program 

79.169 

(0.673) 

*** 

77.851 

80.487 


Difference 

1.426 

(0.924) 


-0.386 

3.237 


Control 

84.040 

(0.907) 

*** 

82.261 

85.818 

Week 16 

Program 

86.671 

(0.845) 

*** 

85.015 

88.327 


Difference 

2.632 

(1.183) 

* 

0.312 

4.951 
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Table B.1.57. Model predicting benchmark-moderated Fall Kindergarten grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Alternative treatment 

0.475 

(0.096) 

*** 

Ability group 2 

1.199 

(0.093) 

*** 

Ability group 3 

1.941 

(0.100) 

*** 

Program 

-0.371 

(0.092) 

*** 

Time b 

0.182 

(0.015) 

*** 

Program xtime 

0.263 

(0.020) 

*** 

Time squared 

-0.002 

(0.001) 

* 

Program xtime squared 

-0.011 

(0.001) 

*** 

Program x ability group 2 

0.194 

(0.126) 


Program x ability group 3 

0.032 

(0.134) 


Ability group 2 xtime 

-0.013 

(0.020) 


Ability group 3 xtime 

0.012 

(0.021) 


Ability group 2 x program x time 

- 0.111 

(0.027) 

*** 

Ability group 3 x program x time 

-0.155 

(0.029) 

*** 

Ability group 2 x time squared 

0.000 

(0.001) 


Ability group 3 x time squared 

-0.003 

(0.001) 

* 

Ability group 2 x program x time squared 

0.004 

(0.001) 

* 

Ability group 3 x program x time squared 

0.006 

(0.002) 

*** 

Intercept 

-0.365 

(0.078) 

*** 

Observations 




Weeks 

4474 



Students 

359 



Pairs 

193 



Schools 

17 




Notes: a: outcome is natural log ofassessmentscore + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p< 0.001 

Kindergarten sample includes 116 Low, 134 Middle, and 109 High Benchmark students 
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Table B.1.58. Model predicting benchmark-moderated Fall first grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Alternative treatment 

-0,040 

(0,022) 


Ability group 2 

0,471 

(0,026) 

*** 

Ability group 3 

0,607 

(0,028) 

*** 

Program 

-0,021 

(0,024) 


Time b 

0,070 

(0,001) 

*** 

Time squared 

0,017 

(0,002) 

*** 

Program x ability group 2 

-0,029 

(0,033) 


Program x ability group 2 

0,007 

(0,034) 


Ability group 2 xtime 

-0,015 

(0,002) 

*** 

Ability group 3 xtime 

-0,021 

(0,002) 

*** 

Ability group 2 x program x time 

-0,012 

(0,003) 

*** 

Ability group 3 x program x time 

-0,014 

(0,003) 

*** 

Intercept 

2.796 

(0,021) 

*** 

Observations 




Weeks 

5078 



Students 

409 



Pairs 

215 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
* p <0.05 

**p<0.01 
***p <0.001 

Firstgrade sample includes 129 Low, 154 Middle, and 126 High Benchmark students 
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Table B.1.59. Model predicting benchmark-moderated Fall third grade students' weekly assessments 3 


Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Alternative treatment 

-0.087 

(0.019) 

*** 

Ability group 2 

0.411 

(0.021) 

*** 

Ability group 3 

0.540 

(0.022) 

*** 

Program 

-0.049 

(0.019) 

** 

Time b 

0.033 

(0.001) 

*** 

Time squared 

0.007 

(0.001) 

*** 

Program x ability group 2 

-0.038 

(0.024) 


Program x ability group 2 

0.011 

(0.025) 


Ability group 2 xtime 

-0.007 

(0.001) 

*** 

Ability group 3 xtime 

-0.014 

(0.001) 

*** 

Ability group 2 x program x time 

-0.003 

(0.002) 


Ability group 3 x program x time 

-0.002 

(0.002) 


Intercept 

3.679 

(0.018) 

*** 

Observations 




Weeks 

3631 



Students 

308 



Pairs 

168 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p<0.01 
***p< 0.001 

Third grade sample includes 89 Low, 113 Middle, and 106 High Benchmark students 
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Table B.1.60. Estimated scores for Kindergarten Low Benchmark program participants and control students 


Kindergarten (N=116) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

-0.131 

(0.060) 

* 

-0.249 

-0.013 

Week 1 

Program 

-0.261 

(0.050) 

*** 

-0.359 

-0.164 


Difference 

-0.130 

(0.063) 

* 

-0.254 

-0.006 


Control 

0.467 

(0.083) 

*** 

0.304 

0.630 

Week 4 

Program 

1.336 

(0.132) 

*** 

1.078 

1.595 


Difference 

0.870 

(0.114) 

*** 

0.646 

1.094 


Control 

1.407 

(0.135) 

*** 

1.143 

1.671 

Week 7 

Program 

4.935 

(0.333) 

*** 

4.283 

5.587 


Difference 

3.528 

(0.286) 

*** 

2.967 

4.089 


Control 

2.841 

(0.216) 

*** 

2.418 

3.265 

Week 10 

Program 

11.107 

(0.681) 

*** 

9.771 

12.443 


Difference 

8.266 

(0.599) 

*** 

7.092 

9.439 


Control 

4.962 

(0.331) 

*** 

4.313 

5.611 

Week 13 

Program 

18.832 

(1.105) 

*** 

16.665 

20.998 


Difference 

13.870 

(0.972) 

*** 

11.965 

15.774 


Control 

7.997 

(0.559) 

*** 

6.903 

9.092 

Week 16 

Program 

25.085 

(1.625) 

*** 

21.901 

28.269 


Difference 

17.088 

(1.478) 

*** 

14.191 

19.984 
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Table B.1.61. Estimated scores for Kindergarten Middle Benchmark program participants and control students 


Kindergarten (N=134) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

1.846 

(0.181) 

*** 

1.491 

2.202 

Week 1 

Program 

1.637 

(0.166) 

*** 

1.312 

1.962 


Difference 

-0.210 

(0.201) 


-0.604 

0.185 


Control 

3.631 

(0.244) 

*** 

3.153 

4.109 

Week 4 

Program 

5.082 

(0.318) 

*** 

4.460 

5.705 


Difference 

1.451 

(0.294) 

*** 

0.875 

2.028 


Control 

6.341 

(0.383) 

*** 

5.591 

7.092 

Week 7 

Program 

11.058 

(0.624) 

*** 

9.835 

12.282 


Difference 

4.717 

(0.552) 

*** 

3.636 

5.798 


Control 

10.338 

(0.593) 

*** 

9.176 

11.501 

Week 10 

Program 

19.544 

(1.067) 

*** 

17.452 

21.635 


Difference 

9.205 

(0.938) 

*** 

7.366 

11.045 


Control 

16.061 

(0.883) 

*** 

14.331 

17.791 

Week 13 

Program 

29.080 

(1.544) 

*** 

26.053 

32.106 


Difference 

13.018 

(1.349) 

*** 

10.374 

15.662 


Control 

24.012 

(1.460) 

*** 

21.151 

26.873 

Week 16 

Program 

36.849 

(2.177) 

*** 

32.583 

41.115 


Difference 

12.837 

(2.095) 

*** 

8.731 

16.942 
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Table B.1.62. Estimated scores for Kindergarten High Benchmark program participants and control students 


Kindergarten (N=109) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

5.106 

(0.430) 

*** 

4.264 

5.949 

Week 1 

Program 

3.619 

(0.314) 

*** 

3.004 

4.234 


Difference 

-1.487 

(0.453) 

** 

-2.376 

-0.599 


Control 

9.250 

(0.581) 

*** 

8.111 

10.388 

Week 4 

Program 

8.963 

(0.543) 

*** 

7.899 

10.027 


Difference 

-0.287 

(0.590) 


-1.444 

0.870 


Control 

14.947 

(0.896) 

*** 

13.191 

16.703 

Week 7 

Program 

17.245 

(0.977) 

*** 

15.330 

19.160 


Difference 

2.298 

(0.971) 

* 

0.394 

4.202 


Control 

21.997 

(1.303) 

*** 

19.443 

24.550 

Week 10 

Program 

27.368 

(1.528) 

*** 

24.372 

30.364 


Difference 

5.371 

(1.485) 

*** 

2.461 

8.281 


Control 

29.738 

(1.717) 

*** 

26.373 

33.103 

Week 13 

Program 

36.448 

(1.988) 

*** 

32.551 

40.346 


Difference 

6.711 

(1.917) 

*** 

2.954 

10.468 


Control 

37.081 

(2.395) 

*** 

32.386 

41.775 

Week 16 

Program 

40.972 

(2.486) 

*** 

36.100 

45.845 


Difference 

3.892 

(2.730) 


-1.458 

9.242 
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Table B.1.63. Estimated scores for first grade Low Benchmark program participants and control students 


FirstGrade (N=129) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

16.490 

(0.351) 

*** 

15.802 

17.178 

Week 1 

Program 

16.498 

(0.344) 

*** 

15.824 

17.172 


Difference 

0.008 

(0.395) 


-0.766 

0.781 


Control 

20.553 

(0.382) 

*** 

19.804 

21.302 

Week 4 

Program 

21.720 

(0.397) 

*** 

20.941 

22.498 


Difference 

1.167 

(0.408) 

** 

0.366 

1.967 


Control 

25.560 

(0.433) 

*** 

24.710 

26.409 

Week 7 

Program 

28.500 

(0.479) 

*** 

27.562 

29.438 


Difference 

2.940 

(0.444) 

*** 

2.069 

3.811 


Control 

31.730 

(0.525) 

*** 

30.700 

32.759 

Week 10 

Program 

37.303 

(0.616) 

*** 

36.096 

38.510 


Difference 

5.574 

(0.552) 

*** 

4.492 

6.655 


Control 

39.333 

(0.686) 

*** 

37.989 

40.677 

Week 13 

Program 

48.734 

(0.849) 

*** 

47.069 

50.398 


Difference 

9.401 

(0.794) 

*** 

7.845 

10.957 


Control 

48.702 

(0.944) 

*** 

46.853 

50.551 

Week 16 

Program 

63.576 

(1.230) 

*** 

61.165 

65.986 


Difference 

14.874 

(1.227) 

*** 

12.469 

17.278 
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Table B.1.64. Estimated scores for first grade Middle Benchmark program participants and control students 


FirstGrade (N=154) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

26.596 

(0.497) 

*** 

25.622 

27.570 

Week 1 

Program 

25.488 

(0.462) 

*** 

24.582 

26.394 


Difference 

-1.108 

(0.548) 

* 

-2.182 

-0.033 


Control 

31.487 

(0.514) 

*** 

30.479 

32.494 

Week 4 

Program 

30.680 

(0.490) 

*** 

29.720 

31.641 


Difference 

-0.806 

(0.528) 


-1.842 

0.230 


Control 

37.244 

(0.554) 

*** 

36.157 

38.330 

Week 7 

Program 

36.890 

(0.541) 

*** 

35.829 

37.952 


Difference 

-0.353 

(0.533) 


-1.397 

0.691 


Control 

44.021 

(0.642) 

*** 

42.763 

45.278 

Week 10 

Program 

44.318 

(0.640) 

*** 

43.063 

45.572 


Difference 

0.297 

(0.614) 


-0.906 

1.500 


Control 

51.999 

(0.803) 

*** 

50.424 

53.574 

Week 13 

Program 

53.201 

(0.814) 

*** 

51.606 

54.795 


Difference 

1.202 

(0.825) 


-0.416 

2.820 


Control 

61.391 

(1.062) 

*** 

59.309 

63.473 

Week 16 

Program 

63.825 

(1.087) 

*** 

61.694 

65.956 


Difference 

2.434 

(1.196) 

* 

0.090 

4.778 
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Table B.1.65. Estimated scores for first grade High Benchmark program participants and control students 


FirstGrade (N=126) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

30.446 

(0.605) 

*** 

29.259 

31.632 

Week 1 

Program 

30.251 

(0.588) 

*** 

29.099 

31.402 


Difference 

-0.195 

(0.700) 


-1.568 

1.178 


Control 

35.417 

(0.606) 

*** 

34.229 

36.606 

Week 4 

Program 

35.584 

(0.601) 

*** 

34.405 

36.763 


Difference 

0.167 

(0.656) 


-1.120 

1.453 


Control 

41.175 

(0.634) 

*** 

39.932 

42.418 

Week 7 

Program 

41.828 

(0.643) 

*** 

40.568 

43.087 


Difference 

0.653 

(0.642) 


-0.605 

1.910 


Control 

47.843 

(0.720) 

*** 

46.431 

49.255 

Week 10 

Program 

49.137 

(0.741) 

*** 

47.685 

50.588 


Difference 

1.294 

(0.722) 


-0.122 

2.709 


Control 

55.565 

(0.897) 

*** 

53.808 

57.323 

Week 13 

Program 

57.693 

(0.927) 

*** 

55.877 

59.510 


Difference 

2.128 

(0.962) 

* 

0.243 

4.013 


Control 

64.509 

(1.185) 

*** 

62.186 

66.831 

Week 16 

Program 

67.710 

(1.227) 

*** 

65.306 

70.115 


Difference 

3.202 

(1.382) 

* 

0.493 

5.910 
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Table B.1.66. Estimated scores for third grade Low Benchmark program participants and control students 


Third Grade (N=89) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

39.525 

(0.691) 

*** 

38.170 

40.879 

Week 1 

Program 

38.208 

(0.657) 

*** 

36.921 

39.495 


Difference 

-1.317 

(0.706) 


-2.701 

0.068 


Control 

43.724 

(0.709) 

*** 

42.335 

45.113 

Week 4 

Program 

43.131 

(0.689) 

*** 

41.781 

44.481 


Difference 

-0.593 

(0.684) 


-1.934 

0.747 


Control 

48.359 

(0.747) 

*** 

46.894 

49.824 

Week 7 

Program 

48.672 

(0.742) 

*** 

47.218 

50.126 


Difference 

0.313 

(0.691) 


-1.042 

1.667 


Control 

53.474 

(0.818) 

*** 

51.870 

55.077 

Week 10 

Program 

54.908 

(0.828) 

*** 

53.286 

56.530 


Difference 

1.434 

(0.756) 


-0.047 

2.915 


Control 

59.119 

(0.931) 

*** 

57.294 

60.944 

Week 13 

Program 

61.928 

(0.958) 

*** 

60.050 

63.806 


Difference 

2.809 

(0.902) 

** 

1.041 

4.576 


Control 

65.349 

(1.095) 

*** 

63.203 

67.496 

Week 16 

Program 

69.829 

(1.145) 

*** 

67.585 

72.072 


Difference 

4.480 

(1.142) 

*** 

2.242 

6.717 
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Table B.1.67. Estimated scores for third grade Middle Benchmark program participants and control students 


Third Grade (N=113) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

59.702 

(0.836) 

*** 

58.065 

61.340 

Week 1 

Program 

55.363 

(0.761) 

*** 

53.871 

56.855 


Difference 

-4.339 

(0.831) 

*** 

-5.968 

-2.711 


Control 

64.677 

(0.837) 

*** 

63.036 

66.318 

Week 4 

Program 

60.649 

(0.775) 

*** 

59.130 

62.169 


Difference 

-4.028 

(0.779) 

*** 

-5.554 

-2.501 


Control 

70.059 

(0.863) 

*** 

68.368 

71.750 

Week 7 

Program 

66.431 

(0.811) 

*** 

64.843 

68.020 


Difference 

-3.628 

(0.760) 

*** 

-5.118 

-2.138 


Control 

75.883 

(0.923) 

*** 

74.074 

77.692 

Week 10 

Program 

72.756 

(0.877) 

*** 

71.036 

74.475 


Difference 

-3.127 

(0.804) 

*** 

-4.703 

-1.551 


Control 

82.183 

(1.027) 

*** 

80.169 

84.197 

Week 13 

Program 

79.673 

(0.985) 

*** 

77.742 

81.604 


Difference 

-2.510 

(0.934) 

** 

-4.341 

-0.680 


Control 

89.000 

(1.183) 

*** 

86.681 

91.319 

Week 16 

Program 

87.239 

(1.142) 

*** 

85.000 

89.477 


Difference 

-1.761 

(1.156) 


-4.027 

0.505 
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Table B.1.68. Estimated scores for third grade High Benchmark program participants and control students 


Third Grade (N=106) 

Estimated Score 

SE 

Sig. 

Confidence Interval 


Control 

67.621 

(1.070) 

*** 

65.525 

69.718 

Week 1 

Program 

66.043 

(0.941) 

*** 

64.199 

67.888 


Difference 

-1.578 

(1.074) 


-3.683 

0.528 


Control 

71.712 

(1.039) 

*** 

69.676 

73.749 

Week 4 

Program 

71.080 

(0.943) 

*** 

69.232 

72.928 


Difference 

-0.633 

(0.985) 


-2.563 

1.298 


Control 

76.047 

(1.041) 

*** 

74.007 

78.088 

Week 7 

Program 

76.495 

(0.969) 

*** 

74.595 

78.395 


Difference 

0.447 

(0.941) 


-1.396 

2.291 


Control 

80.641 

(1.090) 

*** 

78.505 

82.777 

Week 10 

Program 

82.316 

(1.031) 

*** 

80.295 

84.337 


Difference 

1.675 

(0.976) 


-0.237 

3.588 


Control 

85.508 

(1.196) 

*** 

83.165 

87.852 

Week 13 

Program 

88.575 

(1.137) 

*** 

86.346 

90.805 


Difference 

3.067 

(1.114) 

** 

0.883 

5.251 


Control 

90.666 

(1.363) 

*** 

87.994 

93.338 

Week 16 

Program 

95.304 

(1.294) 

*** 

92.768 

97.841 


Difference 

4.638 

(1.357) 

*** 

1.978 

7.299 


APPENDIX B.l: FALL-WINTER MODELS AND EFFECTS TABLES 


PAGE 80 



The Corporation for National and Community Service | 2014 


Appendix B.2: Findings from Robustness Analysis 

The evaluation team also tested the variability of the program effect at the assigned pair level. To do this, we 
estimated a simpler model with a linear trajectory and a random effect for the impact of program participation on 
trajectory, z. 

l n [TW + 1] = ^oyk + n ujk w + P r M V k 
where 

ft Oijk ~ Poo jk + Po 1 Jk + 1 ] + Po 2 jk a tjk + PoijkPijk + e tjk + U jk + V k 

and 

ft Ujk = Ao + PlljkPijk 

and 

P\ 1 jk ~ +Z jk 

where, ^i 0 is the average of the pair-specific differences in the trajectory based on program participation and z is 
the difference between the pair-specific difference in the trajectory and the average. 

After the model was estimated, we then predicted the random effect. This allowed us to estimate a pair-specific 
effect. We then estimated the mean and confidence interval of this effect. If the confidence interval did not include 0, 
we found support for program effects. We also calculated the proportion of effects that were positive as a measure of 
robustness. 

Table B .2.1 showcases the results of this analysis. As expected, we found average positive effects on all simple 
trajectories (noted as average pair difference from model in the table). We also found non-zero variance of this effect 
at the pair level for all grades except third (noted as variance of difference in the table). 

When we computed the pair specific effect from the model, we found the mean pair effects were consistent with the 
model estimated mean effect. These means were, on average, statistically different than zero. 

Finally, looking at the proportion of pairs with an estimated positive effect, we find that 92 percent of kindergartener 
pairs, 97 percent of first graders, 79 percent of second graders, and 70 percent of third graders, showed a positive 
effect on a simple trajectory. 
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Table B.2.1. Results from Robustness Analysis 



Kindergarten 

(N=359) 

First Grade 
(N=409) 

Second Grade 
(N=265) 

Third Grade 
(N=308) 

Average estimated pair difference 

0.0390 

0.0083 

0.0053 

0.0040 

SE 

(0.0033) 

(0.0013) 

(0.0013) 

(0.0008) 

Lower bound 95% Cl 

0.0324 

0.0058 

0.0027 

0.0025 

Higher bound 95% Cl 

0.0455 

0.0108 

0.0080 

0.0056 

Variance of difference 

0.0011 

0.0001 

O.OOOl 

O.OOOl 

SE 

(0.0002) 

(0.0000) 

(0.0000) 

(0.0000) 

Lower bound 95% Cl 

0.0008 

0.0001 

0.0001 

0.0001 

Higher bound 95% Cl 

0.0015 

0.0002 

0.0002 

0.0001 

Average predicted pair difference 

0.0359 

0.0085 

0.0055 

0.0042 

SE 

(0.0015) 

(0.0005) 

(0.0005) 

(0.0006) 

Lower bound 95% Cl 

0.0328 

0.0075 

0.0045 

0.0030 

Higher bound 95% Cl 

0.0389 

0.0094 

0.0065 

0.0054 

Proportion predicted positive effect 

0.9585 

0.9349 

0.8811 

0.6964 

SE 

(0.0144) 

(0.0169) 

(0.0272) 

(0.0356) 

Lower bound 95% Cl 

0.9188 

0.8927 

0.8162 

0.6220 

Higher bound 95% Cl 

0.9793 

0.9612 

0.9252 

0.7618 


Notes: Standard errors in parentheses, confidence intervals in italics 
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Appendix C: IES What Works Clearinghouse Analysis 

The WhatWorks Clearinghouse (WWC), an initiative ofthe Institute of Education Sciences atthe U.S. Department of 
Education, is the leading resource in the field of education that independently identifies and evaluates scientific 
evidence of effectiveness of educational practices, programs and policies. 2 Given the size ofthe MRC program within 
the state of Minnesota and the growing interest in replicating the MRC model in other states, we anticipate WWC 
interestin reviewing the results ofthe MRC K-3 impact evaluation. As such, we present here an analysis ofthe MRC 
K-3 impact evaluation data that adheres to WWC review criteria. 

For the WWC analysis, the evaluation team conducted four separate grade-specific regression analyses comparing 
program and control group students' literacy outcomes as measured by their Winter benchmark scores. In addition, 
using Analysis of Covariance (ANCOVA) models, we tested the effect of treatment on Winter benchmark scores that 
included the Fall benchmark score as a covariate. The results ofthe WWC analysis confirm the statistically significant 
and positive impact of the MRC program on Kindergarten and first grade students' literacy skills. No statistically 
significant impact was found for second and third grade students. Below, we outline details ofthe WWC analysis. 

Outcomes 

As described in Section II. D, the evaluation used outcome data collected by the MRC program using the AIMSweb 
assessment system. 3 Three specific literacy skills that are appropriate for particular grade levels were assessed 
using AIMSweb: 1) letter sound fluency (Kindergarten), 2) nonsense word fluency (first grade), and 3) oral reading 
fluency (i.e., R-CBM; second and third grades). The AIMSweb assessments, published by Pearson, have been 
shown to have good measurement properties in terms of retest reliability for R-CBM, >.90 and letter sound fluency 
>.82; and alternate form reliability for nonsense word fluency >,74. 4 Depending on the criterion referenced test and 
time of administration, criterion validity ranged from ,58 to .72 for letter sound fluency in kindergarten, .43 to .72 for 
nonsense word fluency in firstgrade, and .60 to .81 for R-CBM in 2 nd and 3 rd grade. 5 Since the outcome metrics 
varied by grade level, our analysis is presented as a separate study for each grade. 


2 http://ie5.ed.gov/ncee/wwc/default.a5px 

3 http://www.aimsweb.com/ 

4 http://www.aimsweb.com/resources/research-articles-and-information 

5 See the AIMSweb Technical Manual (2012) for a detailed report on each outcome measure’s reliability and validity. 
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Randomization 

The evaluation implemented a randomized controlled trial, experimental design where randomization to the program 
and control conditions occurred at the level of the student. Because students were randomized, it is not necessary to 
demonstrate equivalence between the program and control groups a priori. Likewise, student level randomization 
eliminates the need to consider the effects of school clustering on uncertainty. 

Attrition 

For each grade K-3, we demonstrate low levels of attrition assuming the "liberal" standards outlined in the WWC 
Evidence Review Protocol for Early Childhood interventions (Version 2). 6 The liberal standards are appropriate 
because in early childhood interventions the mechanism of attrition is unlikely due to program assignment. Any 
combination of overall attrition and differential attrition in the lower, left corner (dark gray zone) of Figure C.l below is 
considered low attrition. 

Figure C.l: Acceptable Levels of Overall Attrition by Differential Attrition 

161 
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Overall Attrition Rate 

In total, the evaluation team randomized 1,530 students to either the program or control conditions based on fall 
assessment scores. 7 Table C.l below shows the number of students in each condition by grade. 

6 http://ies.ed.gov/ncee/wwc/pdf/reference resources/ece protocol v2.0.pdf 

7 See section III.D fora detailed description of the randomization process 
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Table C.l. Number of Students Assigned to and Retained in Each Condition by Grade 


Grade 

Assigned Program 
(Retained/Analytic N) 

Assigned Control 
(Retained/Analytic N) 

Total 

(Retained/Analytic N) 

Kindergarten 

207 (144) 

194 (150) 

401 (294) 

First Grade 

228 (197) 

222 (183) 

450 (380) 

Second Grade 

170 (140) 

148(114) 

318 (254) 

Third Grade 

196 (169) 

165 (142) 

361(311) 

Total 

801 (650) 

729 (589) 

1,530 (1,239) 


A number of students (291) were not available for Winter benchmark testing. This attrition poses a risk to the 
experimental results if the overall rate is too high or the difference in the rate between program and control groups is 
too large. Overall attrition was calculated as: 

Randomized Sample 

Overall = X= x 100 

Analytic Sample 

The differential attrition is the absolute value of the difference in the attrition rates between those assigned to the 
program group and those assigned to the control group. Differential attrition was calculated as: 


Differential = Y 


Randomized Sample (Program) 

Randomized Sample (Control) 

Analytic Sample (Program) 

Analytic Sample (Control) 


Table C. 2 below lists the overall attrition rate (the X axis in Figure C.l above) and the differential attrition rate (the Y 
axis in Figure C.l above) by grade. 


Table C.2. Overall and Differential Attrition Rates by Grade 


Grade 

Overall Attrition (X 
Axis) 

Attrition for Program 
Group (A P ) 

Attrition for Control 
Group (Ac) 

Differential Attrition 
(Absolute Value; Y 
Axis) 

Kindergarten 

27 

30 

23 

7 

First Grade 

16 

14 

18 

4 

Second Grade 

20 

18 

23 

5 

Third Grade 

14 

14 

14 

0 

Total 

19 

19 

19 

0 


When plotting the combination of overall and differential attrition on the WWC attrition guide, illustrated in Figure C.l 
above, we find acceptably low attrition levels for each grade (i.e„ levels of overall attrition by differential attrition rates 
are in the green area of the figure). 
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Effect 

Table C. 3 below presents the results in effect-size units (Hedges' g) and the original assessment metric of four 
separate grade specific analyses comparing program and control group students' literacy outcomes measure by 
their Winter benchmark scores. The table also provides the results of Analysis of Covariance (ANCOVA) models 
testing the effect of treatment with a regression that also included the Fall benchmark score as a covariate. 


Table C. 3. Effect of MRC on Student Literacy Outcomes by Grade 



Effect 

Mean Difference 

ANCOVA 

Grade 

Hedges’ g 

Original Metric 

Kindergarten 

0.52*** 

10.11*** 

10.08*** 

First Grade 

0.39*** 

6.85*** 

6.85*** 

Second Grade 

-0.03 

-0.54 

1.81 

Third Grade 

-0.10 

-2.09 

0.96 


***p<0.001 


Table C. 3 shows a substantial effect size for Kindergarten that is also highly significant; an average difference of 
approximately 10.1 letter sounds between program and control group students. A highly significant substantial effect 
size was also found for first grade students; an average difference of 6.9 letters sounds within nonsense words 
between program and control group students. In the second and third grades, the evaluation team observed a slight 
negative mean difference, with an effect size that is less than meaningful (< 0.2) and not statistically significant. Of 
note, when we control for the Fall test score, the direction of the effect is reversed, but remains statistically not 
significant. 


APPENDIX C: IES WHAT WORKS CLEARINGHOUSE ANALYSIS 


PAGE 86 



The Corporation for National and Community Service | 2014 


Appendix D: Full Year Models and Effects Tables 


Table D.l Estimated week-to-week effects of program for Kindergarten student 



Week 


Effect 

SE (Effect) 

Sig. 

Cl (95%) 

0 

to 

1 

2.713 

(0.161) 

*** 

2.398 

3.029 

1 

to 

2 

2.592 

(0.158) 

*** 

2.282 

2.901 

2 

to 

3 

2.411 

(0.150) 

*** 

2.117 

2.706 

3 

to 

4 

2.182 

(0.138) 

*** 

1.910 

2.453 

4 

to 

5 

1.915 

(0.125) 

*** 

1.671 

2.159 

5 

to 

6 

1.624 

(0.111) 

*** 

1.406 

1.842 

6 

to 

7 

1.321 

(0.100) 

*** 

1.125 

1.517 

7 

to 

8 

1.020 

(0.093) 

*** 

0.837 

1.203 

8 

to 

9 

0.731 

(0.091) 

*** 

0.552 

0.911 

9 

to 

10 

0.465 

(0.093) 

*** 

0.282 

0.648 

10 

to 

11 

0.228 

(0.097) 

* 

0.037 

0.418 

11 

to 

12 

0.025 

(0.101) 


-0.174 

0.224 

12 

to 

13 

-0.139 

(0.104) 


-0.343 

0.066 

13 

to 

14 

-0.263 

(0.106) 

* 

-0.470 

-0.056 

14 

to 

15 

-0.347 

(0.106) 

** 

-0.553 

-0.140 

15 

to 

16 

-0.390 

(0.104) 

*** 

-0.594 

-0.187 

16 

to 

17 

-0.396 

(0.102) 

*** 

-0.595 

-0.196 

17 

to 

18 

-0.363 

(0.099) 

*** 

-0.558 

-0.169 

18 

to 

19 

-0.295 

(0.098) 

** 

-0.488 

-0.103 

19 

to 

20 

-0.192 

(0.099) 


-0.387 

0.003 

20 

to 

21 

-0.055 

(0.104) 


-0.259 

0.150 

21 

to 

22 

0.119 

(0.114) 


-0.105 

0.344 

22 

to 

23 

0.332 

(0.130) 

* 

0.076 

0.587 

23 

to 

24 

0.587 

(0.154) 

*** 

0.286 

0.888 

24 

to 

25 

0.891 

(0.185) 

*** 

0.528 

1.255 

25 

to 

26 

1.255 

(0.227) 

*** 

0.810 

1.700 

26 

to 

27 

1.692 

(0.282) 

*** 

1.139 

2.245 

27 

to 

28 

2.221 

(0.354) 

*** 

1.527 

2.915 


*p <0.05 

** p < 0.01 
***p <0.001 

Kindergarten sample includes 368 students. 
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Table D.2. Model predicting Kindergarten student weekly assessments 3 as a function of cumulative weekly 
sessions 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Program 

-0.014 

(0.034) 


Spring 

-0.237 

(0.015) 

*** 

Time b 

0.184 

(0.002) 

*** 

Time Squared 

-0.003 

(0.000) 

*** 

Number of sessions 

0.065 

(0.003) 

*** 

Number of session squared 

-0.002 

(0.000) 

*** 

Intercept 

0.858 

(0.048) 

*** 

Observations 




Weeks 

10399 



Students 

368 



Schools 

17 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

** p < 0.01 
***p< 0.001 

Kindergarten sample includes 368 students. 
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Table D.3. Estimated week-to-week effects of program for first grade students 


Fall Metric Spring Metric 



Week 


Effect 

SE (Effect) 

Sig. 

Cl (95%) 

Effect 

SE (Effect) 

Sig. 

Cl (95%) 

0 

to 

1 

0.529 

(0.058) 

*** 

0.415 

0.644 

0.230 

(0.022) 

*** 

0.186 

0.273 

1 

to 

2 

0.536 

(0.060) 

*** 

0.418 

0.653 

0.231 

(0.022) 

*** 

0.187 

0.275 

2 

to 

3 

0.542 

(0.061) 

*** 

0.422 

0.663 

0.233 

(0.023) 

*** 

0.188 

0.278 

3 

to 

4 

0.549 

(0.063) 

*** 

0.426 

0.672 

0.235 

(0.023) 

*** 

0.189 

0.280 

4 

to 

5 

0.556 

(0.064) 

*** 

0.430 

0.682 

0.236 

(0.023) 

*** 

0.190 

0.282 

5 

to 

6 

0.563 

(0.066) 

*** 

0.433 

0.692 

0.238 

(0.024) 

*** 

0.192 

0.284 

6 

to 

7 

0.570 

(0.068) 

*** 

0.437 

0.702 

0.239 

(0.024) 

*** 

0.193 

0.286 

7 

to 

8 

0.577 

(0.069) 

*** 

0.441 

0.713 

0.241 

(0.024) 

*** 

0.194 

0.289 

8 

to 

9 

0.584 

(0.071) 

*** 

0.445 

0.723 

0.243 

(0.025) 

*** 

0.195 

0.291 

9 

to 

10 

0.591 

(0.073) 

*** 

0.449 

0.734 

0.244 

(0.025) 

*** 

0.196 

0.293 

10 

to 

11 

0.599 

(0.074) 

*** 

0.453 

0.745 

0.246 

(0.025) 

*** 

0.197 

0.295 

11 

to 

12 

0.606 

(0.076) 

*** 

0.457 

0.756 

0.248 

(0.026) 

*** 

0.198 

0.298 

12 

to 

13 

0.614 

(0.078) 

*** 

0.461 

0.767 

0.249 

(0.026) 

*** 

0.199 

0.300 

13 

to 

14 

0.621 

(0.080) 

*** 

0.465 

0.778 

0.251 

(0.026) 

*** 

0.200 

0.302 

14 

to 

15 

0.629 

(0.082) 

*** 

0.469 

0.789 

0.253 

(0.027) 

*** 

0.201 

0.305 

15 

to 

16 

0.637 

(0.084) 

*** 

0.473 

0.801 

0.255 

(0.027) 

*** 

0.202 

0.307 


*p <0.05 

**p <0.01 
*** p <0.001 


Firstgrade sample includes 393 students. 
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Table D.4. Model predicting first grade students' weekly assessments 3 as a function of cumulative weekly sessions 



Effect 

SE (Effect) 

Sig. 

Effect 

SE (Effect) 

Sig. 

Fixed Effects 







Program 

-0.020 

(0.015) 


-0.014 

(0.028) 


Time b 

0.057 

(0.001) 

*** 

0.036 

(0.000) 

*** 

Number of sessions 

0.012 

(0.001) 

*** 

0.007 

(0.001) 

*** 

Intercept 

3.185 

(0.016) 

*** 

2.475 

(0.035) 

*** 

Observations ' 1 







Weeks 


6607 



6607 


Students 


393 



393 


Schools 


21 






Notes: a: outcome is natural log ofassessmentscore + 1; b: weeks since 9-1-2012; c: Sample sizes for Fall firstgrade are those forSpring firstgrade, 
* p <0.05 

**p <0.01 
***p <0.001 

Firstgrade sample includes 393 students. 
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Table D.5. Estimated week-to-week effects of program for second grade students 


Week 

Effect 

SE (Effect) 

Sig. 

Cl (95%) 

0 

to 

1 

0.252 

( 0 . 019 ) 

*** 

0.214 

0.290 

1 

to 

2 

0.253 

( 0 . 020 ) 

*** 

0.214 

0.291 

2 

to 

3 

0.254 

( 0 . 020 ) 

*** 

0.215 

0.293 

3 

to 

4 

0.255 

( 0 . 020 ) 

*** 

0.216 

0.294 

4 

to 

5 

0.256 

( 0 . 020 ) 

*** 

0.217 

0.296 

5 

to 

6 

0.257 

( 0 . 020 ) 

*** 

0.218 

0.297 

6 

to 

7 

0.259 

( 0 . 020 ) 

*** 

0.219 

0.299 

7 

to 

8 

0.260 

( 0 . 021 ) 

*** 

0.219 

0.300 

8 

to 

9 

0.261 

( 0 . 021 ) 

*** 

0.220 

0.302 

9 

to 

10 

0.262 

( 0 . 021 ) 

*** 

0.221 

0.304 

10 

to 

11 

0.264 

( 0 . 021 ) 

*** 

0.222 

0.305 

11 

to 

12 

0.265 

( 0 . 021 ) 

*** 

0.223 

0.307 

12 

to 

13 

0.266 

( 0 . 022 ) 

*** 

0.224 

0.308 

13 

to 

14 

0.267 

( 0 . 022 ) 

*** 

0.224 

0.310 

14 

to 

15 

0.268 

( 0 . 022 ) 

*** 

0.225 

0.312 

15 

to 

16 

0.270 

( 0 . 022 ) 

*** 

0.226 

0.313 

16 

to 

17 

0.271 

( 0 . 022 ) 

*** 

0.227 

0.315 

17 

to 

18 

0.272 

( 0 . 023 ) 

*** 

0.228 

0.317 

18 

to 

19 

0.274 

( 0 . 023 ) 

*** 

0.229 

0.318 

19 

to 

20 

0.275 

( 0 . 023 ) 

*** 

0.230 

0.320 

20 

to 

21 

0.276 

( 0 . 023 ) 

*** 

0.231 

0.322 

21 

to 

22 

0.277 

( 0 . 023 ) 

*** 

0.231 

0.323 

22 

to 

23 

0.279 

( 0 . 024 ) 

*** 

0.232 

0.325 

23 

to 

24 

0.280 

( 0 . 024 ) 

*** 

0.233 

0.327 

24 

to 

25 

0.281 

( 0 . 024 ) 

*** 

0.234 

0.328 

25 

to 

26 

0.283 

( 0 . 024 ) 

*** 

0.235 

0.330 

26 

to 

27 

0.284 

( 0 . 024 ) 

*** 

0.236 

0.332 

27 

to 

28 

0.285 

( 0 . 025 ) 

*** 

0.237 

0.334 


*p < 0.05 

** p < 0.01 
*** p < 0.001 

Second grade sample includes 267 students. 
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Table D.6. Model predicting second grade students' weekly assessments 3 as a function of cumulative weekly 
sessions 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Program 

-0.039 

(0.019) 

* 

Spring 

0.005 

(0.006) 


Time b 

0.020 

(0.000) 

*** 

Numbersessions 

0.005 

(0.000) 

*** 

Intercept 

3.558 

(0.023) 

*** 

Observations 




Weeks 

7874 



Students 

267 



Schools 

20 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

** p < 0.01 
***p< 0.001 

Second grade sample includes 267 students. 
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Table D.7 Estimated week-to-week effects of program for third grade students 



Week 


Effect 

SE (Effect) 

Sig. 

Cl (95%) 

0 

to 

1 

0.198 

(0.019) 

*** 

0.160 

0.236 

1 

to 

2 

0.199 

(0.019) 

*** 

0.160 

0.237 

2 

to 

3 

0.199 

(0.020) 

*** 

0.161 

0.237 

3 

to 

4 

0.200 

(0.020) 

*** 

0.161 

0.238 

4 

to 

5 

0.200 

(0.020) 

*** 

0.161 

0.239 

5 

to 

6 

0.201 

(0.020) 

*** 

0.162 

0.239 

6 

to 

7 

0.201 

(0.020) 

*** 

0.162 

0.240 

7 

to 

8 

0.202 

(0.020) 

*** 

0.162 

0.241 

8 

to 

9 

0.202 

(0.020) 

*** 

0.162 

0.242 

9 

to 

10 

0.202 

(0.020) 

*** 

0.163 

0.242 

10 

to 

11 

0.203 

(0.020) 

*** 

0.163 

0.243 

11 

to 

12 

0.203 

(0.020) 

*** 

0.163 

0.244 

12 

to 

13 

0.204 

(0.021) 

*** 

0.164 

0.244 

13 

to 

14 

0.204 

(0.021) 

*** 

0.164 

0.245 

14 

to 

15 

0.205 

(0.021) 

*** 

0.164 

0.246 

15 

to 

16 

0.205 

(0.021) 

*** 

0.165 

0.246 

16 

to 

17 

0.206 

(0.021) 

*** 

0.165 

0.247 

17 

to 

18 

0.206 

(0.021) 

*** 

0.165 

0.248 

18 

to 

19 

0.207 

(0.021) 

*** 

0.166 

0.248 

19 

to 

20 

0.208 

(0.021) 

*** 

0.166 

0.249 

20 

to 

21 

0.208 

(0.021) 

*** 

0.166 

0.250 

21 

to 

22 

0.209 

(0.021) 

*** 

0.166 

0.251 

22 

to 

23 

0.209 

(0.022) 

*** 

0.167 

0.251 

23 

to 

24 

0.210 

(0.022) 

*** 

0.167 

0.252 

24 

to 

25 

0.210 

(0.022) 

*** 

0.167 

0.253 

25 

to 

26 

0.211 

(0.022) 

Pp k 

0.168 

0.253 

26 

to 

27 

0.211 

(0.022) 

*** 

0.168 

0.254 

27 

to 

28 

0.212 

(0.022) 

*** 

0.168 

0.255 


*p <0.05 

**p<0.01 
***p< 0.001 

Third grade sample includes 310 students. 
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Table D.8 Model predicting third grade students' weekly assessments 3 as a function of cumulative weekly sessions 



Effect 

SE (Effect) 

Sig. 

Fixed Effects 




Program 

-0.022 

(0.012) 


Spring 

-0.034 

(0.004) 

*** 

Time b 

0.013 

(0.000) 

*** 

Numbersessions 

0.002 

(0.000) 

*** 

Intercept 

4.158 

(0.014) 

*** 

Program 

-0.022 

(0.012) 


Observations 




Weeks 

8788 



Students 

310 



Schools 

21 




Notes: a: outcome is natural log of assessment score + 1; b: weeks since 9-1-2012 
*p <0.05 

**p <0.01 
***p<0.001 

Third grade sample includes 310 students. 
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Table D.9. Multinomial logistic regression model predicting summary outcome of student participants 


Predictor 

Effect 

SE (Effect) 

Sig. 

Lasting effect vs. no effect 

Number of session weeks 

-0.035 

(0.019) 


Assigned program 

2.828 

(0.345) 

*** 

Assigned program x 



*** 

Numberof session weeks 

-0.073 

(0.018) 

*** 

Dual Language Learner 

-0.327 

(0.174) 


Black 

-0.955 

(0.315) 

** 

Asian 

-0.549 

(0.239) 

* 

Hispanic 

-0.863 

(0.387) 

* 

Other race 

-0.598 

(0.553) 


Female 

0.061 

(0.164) 


Intercept 

-0.333 

(0.317) 


Inconsistent effect vs. no effect 

Numberof session weeks 

-0.043 

(0.022) 

* 

Assigned program 

2.493 

(0.370) 

*** 

Assigned program x 



*** 

Numberof session weeks 

-0.053 

(0.027) 

* 

Dual Language Learner 

-0.010 

(0.313) 


Black 

-0.886 

(0.266) 

*** 

Asian 

-0.539 

(0.281) 


Hispanic 

-0.758 

(0.397) 


Other race 

-1.133 

(0.465) 

* 

Female 

0.194 

(0.161) 


Intercept 

-0.802 

(0.169) 

*** 


Notes: 


N =1,348 

pseudo-R-square =0.10 
* p <0.05 

**p <0.01 
***p <0.001 
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Table D.10. Fixed effects regression model predicting students' weekly assessments (spline model) 



Kindergarten 


Second Grade 


Third Grade 


Effect 

SE (Effect) 

Sig 

Effect 

SE (Effect) 

Sig 

Effect 

SE (Effect) 

Sig 

Time 

0.906 

(0.033) 

*** 

1.024 

(0.050) 

*** 

1.094 

(0.063) 

*** 

Time x 

During program 

0.257 

(0.025) 

*** 

0.069 

(0.037) 


-0.108 

(0.048) 

* 

After program 

0.445 

(0.031) 

*** 

0.057 

(0.057) 


-0.103 

(0.063) 


Intercept 

5.902 

(0.323) 

*** 

31.848 

(0.443) 

*** 

63.634 

(0.472) 

*** 

N Weeks 

4,508 



2,847 



3,546 



N Students 

264 



198 



226 



R2 

Within 

0.635 



0.414 



0.328 



Between 

0.471 



0.235 



0.405 



Overall 

0.505 



0.321 



0.376 




Notes: 

*p <0.05 

** p < 0.01 
***p< 0.001 
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Appendix E: Glossary 

AIMline: A student's projected growth trajectory, measured from the most recent benchmark period to the next (e.g.. 
Fall to Winter or Winter to Spring). 

AIMSWeb: AIMSWeb is the web-based assessmenttool used for progress monitoring and data management of all 
Minnesota Reading Corps K-3 students. The AIMSWeb assessments evaluate four critical literacy skills thatare 
appropriate for specific K-3 grade levels and seasons: 1) letter naming fluency (Kindergarten), 2) letter sound fluency 
(FirstGrade -Fall/Winter), 3) nonsense word fluency (FirstGrade - Winter/Spring), and 4) oral reading fluency 
(Second and Third Grades). 

AmeriCorps: AmeriCorps is a national service program run by the Corporation for National and Community Service 
that engages members to serve at nonprofit organizations, public agencies and faith-based organizations nationwide. 
In exchange for their service, AmeriCorps members receive a modest living stipend and Education Award. Minnesota 
Reading Corps is the nation's largest state AmeriCorps program. 

Benchmark: A standard score above which students are considered "on-track" for grade level achievement. MRC 
lists grade and season (l.e., Fall, Winter, Spring) appropriate benchmark scores for each general outcome measure 
(i.e., AIMSWeb & IGIDI). Students' scores on benchmark assessments determine their eligibility for Minnesota 
Reading Corps services and serve as baseline data to determine students' improvements as a result of the program. 

"Big Five": Atthe PreK level, the MRC program focuses on integrating the "Big Five" Early Literacy Predictors into 
all aspects of the daily classroom routine. The "Big Five" for preschool students include conversational skills, 
vocabulary and background knowledge, book and print rules, phonological awareness (i.e., rhyming and alliteration), 
and alphabetic knowledge. 

CNCS: The Corporation for National and Community Service (CNCS) is a federal agency that engages more than 4 
million Americans in service through Senior Corps, AmeriCorps, and the Social Innovation Fund, and leads President 
Obama's national call to service initiative, United We Serve. CNCS is funding the primary federal funder of the 
Minnesota Reading Corps program, and is funding the current evaluation ofthe MRC. 

Community Corps: Community Corps members are embedded in preschool classrooms and collaborate with the 
classroom's lead teacher to help develop children's early literacy skills to prepare for Kindergarten. Community Corps 
members are responsible for enhancing the literacy-rich environment within the classroom, conducting Tier 2 and 3 
interventions and conducting/ tracking progress monitoring for students. 


APPENDIX E: GLOSSARY 


PAGE 97 


The Corporation for National and Community Service | 2014 


Community-based PreK: Community-based PreK programs are run through community-based organizations such 
as community centers. 

DIBELS: Dynamic Indicators of Basic Early Literacy Skills (DIBELS) are a set of assessments used for universal 
screening and progress monitoring in grades K-6. DIBELS assessments help educators identify students who may 
need additional literacy instruction in order to become proficient readers. In the MRC program it is used by some 
schools as alternative assessment to AIMSWeb. 

ELLCO: The Early Language & Literacy Classroom Observation (ELLCO) tool is used to assess five key elements of 
a classroom's literacy environment: classroom structure, curriculum, language environment, books and book reading, 
printand early writing. According to the ELLCO, a "Literacy Rich Classroom" is one thatembeds literacy activities 
among daily routines. 

Head Start: Head Start is a federal PreK program designed to promote school readiness for low-income pre- 
Kindergarten students by enhancing their cognitive, social and emotional development. Through Head Start 
programs, enrolled children and families can also receive health, nutrition and other social support programs 
depending on eligibility. 

IGDI: The Individual Growth and Development Indicators (IGDI) is a set of standardized, individually administered 
assessments that are used to evaluate children's language and emergent literacy skills. IG DIs assess three key 
areas of emergent literacy: (1) Rhyming (Phonological Awareness): (2) Picture Naming (Vocabulary): and (3) 
Alliteration (Phonological Awareness). 

Internal Coach: Individual rained by the Reading Corps to provide on-site literacy supportand oversightto the MRC 
AmeriCorps member. Internal Coaches provide an on-site orientation for the MRC member, develop a daily 
schedule, assist in the implementation of literacy assessments, conduct integrity checks of the assessments and 
interventions, review student data and ensure the member is accurately reporting student data into AIMSWeb. The 
Internal Coach is a school employee, nota MRC member. 

K-3: Kindergarten through third grade 

K-Focus: A Kindergarten-Focused Literacy program in which members provide an additional 20 minutes of tutoring 
every day to small groups of students, typically focused on letter sound fluency. Kindergarten students who are 
served by K-Focus members receive a total of 40 minutes of tutoring everyday. 

Literacy rich schedule: Members in the PreK program work to implement and support a standard instructional 
regime/schedule that focuses on the "Big Five" emergent literacy skills (conversation skills, vocabulary and 
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background knowledge, book and print rules, phonological awareness-rhyming and alliteration, and alphabetic 
knowledge). Members assistthe teaching team in implementing the literacy rich schedule and fostering a literacy rich 
classroom environment as defined by the Early Language and Literacy Classroom Observation (ELLCO), including 
name chart, theme-related books and props in five or more centers, sign-in area, writing center, word wall, etc. 

Master Coach: Provides literacy coaching support to Internal Coaches and AmeriCorps members at multiple sites. 
The MasterCoach schedules regular on-site visits to support and guide the site and its members it fulfilling the MRC 
program goals and ensures fidelity of implementation. The Master Coach provides training to members, conducts 
integrity checks, and reviews students' progress monitoring data. The Master Coach is an experienced literacy 
educatorwho serves as a consultantto MRC. 

Member: A volunteer to who serves in the MRC AmeriCorps program. Member may refer to a volunteer in the K-3 
program, ora PreK Professional Corps or Community Corps. Members deliver the one-on-one tutoring (PreK and K- 
3) and support implementation of the literacy rich schedule in MRC PreK classrooms. In recognition of their service, 
members receive a modest living stipend and Education Award. 

MDE: Minnesota Departmentof Education. 

MRC: The Minnesota Reading Corps (MRC) was started in 2003 to provide reading and literacy tutoring to children in 
PreK programs and students in Kindergarten through third-grade. The goal ofthe program is to ensure that students 
become successful readers and meet reading proficiency targets by the end ofthe third grade. MRC engages 
AmeriCorps members to provide literacy enrichment and tutoring services to PreK students. AmeriCorps members 
serve as one-on-one tutors and provide research-based interventions to both PreK and K-3 students who are just 
below proficiency in reading. As ofthe 2012-2013 school year, more than 1,100 AmeriCorps members implemented 
the program in 652 schools or sites and 184 school districts across the state of Minnesota. 

OnCorps: A web-based database thatstores information aboutstudents receiving MRC services. The database 
records student demographic information, assessment data, the amount of tutoring services students received, and 
other data such as: the number of books sent home, the number of times student's journals were completed, and 
family participation in MRC services 

PreK: Preschool. 

Professional Corps: Professional Corps members are current employees who are in a teaching position at the site. 
This member continues to fulfill their regular teaching responsibilities, butalso incorporates specific MRC strategies 
in their instruction. 
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Program Coordinator: An employee of Minnesota Reading Corps, responsible for providing administrative oversight 
to the Minnesota Reading Corps program on a regional level, including member management, site management, and 
compliance with all AmeriCorps regulations. The Program Coordinator oversees regional recruitment efforts, works 
together with service sites in the interviewing, selection, and placement process for members. 

Progress monitoring: A scientifically-based practice using weekly 1-minute reading tests to assess students' 
academic performance and evaluate the effectiveness of an intervention. Progress monitoring data helps teaching 
teams determine the effectiveness of interventions then make adjustments to instruction to ensure students reach 
their next benchmark target. For K-3, progress monitoring is conducted with all students receiving MRC tutoring each 
week by MRC members using 1-minute AIMSWeb tests. The scores are entered into AIMSWeb and used to create 
graphs that represent each student's progress relative to a specific intervention. ForPre-K, progress monitoring 
occurs monthly, for students receiving Tier 2 and 3 one-to-one intervention services. 

Rtl: Response to Intervention (Rtl) is a practice of academic and behavior interventions designed to identify and 
provide early effective assistance to underperforming students. Research-based interventions are implemented and 
frequent progress monitoring is conducted to assess student response and progress. When students do not make 
progress, increasingly more intense interventions are introduced. 

SEEDS: The SEEDS model is an interactive, skills-focused curriculum based on current research in early childhood 
education, child development, emergent literacy, and effective teaching. S E E DS is a relationship-based professional 
development program that provides a map to help adults intentionally demonstrate behaviors throughout their daily 
interactions with children that enrich academic growth and promote social/emotional well-being. SEEDS interaction 
include the following elements: Sensitivity - Look, listen, and ask questions to become aware of each child's needs, 
thoughts, abilities and feelings: Encouragement - Use intentional affirmations and positive non-verbal 
communication to create a shared positive learning environment: Education - Embed the "Big 5" literacy skills in 
daily routines (vocabulary, conversation, phonological awareness, book and print rules, and letter knowledge): 
Development of Skills Through Doing - Help children explore their world through hands-on learning: Self- 
Image Support- Balance the S E E DS quality interactions to support a child's feeling of being respected and capable. 

ServeMinnesota: State Commission on AmeriCorps programs in Minnesota and responsible in Minnesota Law for 
Minnesota Reading Corps. 

Service hours: The required hours of service AmeriCorps members must complete in order to fulfill their 11 months 
of service to AmeriCorps, and in return receive a living allowance and an education award to pay for college or pay 
back student loans. All full time members, K-3, Community Corps, and Professional Corps, must complete 1700 
hours of service. Part time members must complete 920 hours. Service hours can be fulfilled not only through 
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members' time tutoring or working in the classroom, but also through participation in community and other school 
activities. 

Summer Institute: A multi-day training program conducted over the summer to introduce new and old members, 
Internal Coaches, and Master Coaches to the Reading Corps program. The Institute consists of learning about the 
theories behind the program, the techniques used to implement the program, and the administrative components of 
the program. Education experts train Members and Coaches on all aspects of the program, and also provide time for 
practicing the techniques and the interventions. The Institute is also the time when most members will meet their 
Internal and Master Coaches for the first time. 

Tier 1-3: Tier 1, 2, and 3 are the three "tiers" of a tiered instructional process lying at the core of the Rtl model. 
Student scores on general outcome measure (e.g„ AIMSWeb or IGDI) referenced to specific benchmarks determine 
a student's tier placement. The instruction that is then provided to students is based upon their respective tiers. Tier 1 
students, approximately 75-80% of the population, are at the "Universal Level" and benefit from the standard whole 
class core literacy curriculum. They do not require supplemental instruction. Students who score in Tier 2 range, 15- 
20%, are those whose assessment scores are below the expected levels of achievement (benchmark) and are at risk 
for academic failure but are still above levels considered to indicate a high risk for failure. Tier 2 students typically are 
eligible for supplemental small group instruction. Students whose scores place them into Tier 3, approximately 5-10% 
of students, are considered to be at high risk for academic failure. They are typically offered one-to-one supplemental 
interventions and individualized educational plans. 

*Tier 1 Instruction: In PreK programs, this is instruction that students receive in the general education classroom. It 
includes Reading Corps directed intentional teaching with embedded and explicit instruction. In K-3 programs, this is 
considered the core literacy instruction provided in the classrooms for all students. 

^Tier 2 Instruction: (PreK program term) Provides additional, more intense instruction to children identified as 
needing extra help in targeted skill areas. Tier 2 instruction is in addition to Tier 1 instruction. 

^Tier 3 Instruction: (PreK program term) Provides the most intense intervention approach for children identified as 
needing extra help in a targeted skill area. Tier 3 instruction builds onto Tier 2 instruction by providing more 
individualized and intense instruction. 
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